SI_330_Day_15_NLP

import pandas as pd import nltk

nltk.download('punkt') nltk.download('averaged_perceptron_tagger') nltk.download('maxent_ne_chunker') nltk.download('words')

[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /root/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
[nltk_data] Downloading package maxent_ne_chunker to
[nltk_data]     /root/nltk_data...
[nltk_data]   Unzipping chunkers/maxent_ne_chunker.zip.
[nltk_data] Downloading package words to /root/nltk_data...
[nltk_data]   Unzipping corpora/words.zip.

sentence = "In 2012, Hillary Clinton sent some email messages." tokens = nltk.word_tokenize(sentence) tagged = nltk.pos_tag(tokens) entities = nltk.chunk.ne_chunk(tagged)

!pip install svgling

Collecting svgling
  Downloading svgling-0.3.1-py3-none-any.whl (21 kB)
Collecting svgwrite
  Downloading svgwrite-1.4.1-py3-none-any.whl (66 kB)
     |████████████████████████████████| 66 kB 10.8 MB/s 
Installing collected packages: svgwrite, svgling
Successfully installed svgling-0.3.1 svgwrite-1.4.1
WARNING: You are using pip version 20.1.1; however, version 22.0.3 is available.
You should consider upgrading via the '/root/venv/bin/python -m pip install --upgrade pip' command.

entities

entities.pprint()

(S
  In/IN
  2012/CD
  ,/,
  (PERSON Hillary/NNP Clinton/NNP)
  sent/VBD
  some/DT
  email/NN
  messages/NNS
  ./.)

type(entities)

for entity in entities.subtrees(): if entity.label() == "PERSON": name = [] for leaf in entity.leaves(): name.append(leaf[0]) print(" ".join(name))

Hillary Clinton

sentence = """British Prime Minister Boris Johnson told German Chancellor Olaf Scholz that Western "inaction or underreaction" in response to Russia invading Ukraine would have unthinkable consequences, a spokeswoman for his office said on Thursday. Johnson held calls with Norway's Prime Minister Jonas Gahr Stoere and Germany's Scholz to discuss sanctions after Russian forces invaded Ukraine in a massed assault by land, sea and air.""" tokens = nltk.word_tokenize(sentence) tagged = nltk.pos_tag(tokens) entities = nltk.chunk.ne_chunk(tagged) for entity in entities.subtrees(): if entity.label() == "PERSON": name = [] for leaf in entity.leaves(): name.append(leaf[0]) print(" ".join(name))

Boris Johnson
German Chancellor Olaf Scholz
Ukraine
Johnson
Jonas Gahr Stoere
Scholz

# from https://www.reuters.com/markets/commodities/already-peril-biden-climate-agenda-faces-us-supreme-court-test-2022-02-24/ texts = pd.DataFrame({'text':[ """President Joe Biden's climate-related agenda, already under threat due to congressional failure to pass key legislation, now faces the prospect of a hostile reception at the U.S. Supreme Court that could have lasting consequences on the use of federal power to tackle environmental issues.""", """The court's 6-3 conservative majority, suspicious of broad federal agency power, will weigh at oral arguments next Monday the Environmental Protection Agency's (EPA) authority to regulate greenhouse gas emissions from existing coal- and gas-fired power plants under the landmark Clean Air Act.""", """An eventual ruling restricting EPA authority could hobble the administration's ability to curb the power sector's emissions - representing about a quarter of U.S. greenhouse gases.""", """"Could it be damaging? If it's an adverse decision, of course it could be," John Kerry, the Biden administration's special envoy on climate change, told Reuters.""", """The United States, trailing only China in greenhouse gas emissions, is a crucial player in global efforts to combat climate change.""", """The case before the Supreme Court was brought by Republican-led states led by coal producer West Virginia. Other challengers include coal companies and coal-friendly industry groups. Coal is among the most greenhouse gas-intensive fuels.""", """Democratic-led states and major power companies including Consolidated Edison Inc(ED.N), Exelon Corp (EXC.O) and PG&E Corp (PCG.N) sided with Biden's administration, as did the Edison Electric Institute, an investor-owned utility trade group. The utility industry believes regulatory certainty will help companies devise investment plans.""", """The justices will review the U.S. Court of Appeals for the District of Columbia Circuit's 2021 ruling striking down Republican former President Donald Trump's Affordable Clean Energy rule. That regulation would have imposed limits on a Clean Air Act provision called Section 111 that gives the EPA authority to regulate emissions from existing power plants.""", """The rule proposed by Trump, a supporter of the U.S. coal industry who also questioned climate change science, was meant to replace Democratic former President Barack Obama's Clean Power Plan requiring big cuts in carbon emissions from the power industry."""]})

texts

textobject

President Joe Biden's climate-related agenda, already under threat due to congressional failure to pass key legislation, now faces the prospect of a hostile reception at the U.S. Supreme Court that could have lasting consequences on the use of federal power to tackle environmental issues.11.1%

The court's 6-3 conservative majority, suspicious of broad federal agency power, will weigh at oral arguments next Monday the Environmental Protection Agency's (EPA) authority to regulate greenhouse gas emissions from existing coal- and gas-fired power plants under the landmark Clean Air Act.11.1%

7 others77.8%

President Joe Biden's climate-related agenda, already under threat due to congressional failure to …

The court's 6-3 conservative majority, suspicious of broad federal agency power, will weigh at oral…

An eventual ruling restricting EPA authority could hobble the administration's ability to curb the …

"Could it be damaging? If it's an adverse decision, of course it could be," John Kerry, the Biden a…

The United States, trailing only China in greenhouse gas emissions, is a crucial player in global e…

The case before the Supreme Court was brought by Republican-led states led by coal producer West Vi…

Democratic-led states and major power companies including Consolidated Edison Inc(ED.N), Exelon Cor…

The justices will review the U.S. Court of Appeals for the District of Columbia Circuit's 2021 ruli…

The rule proposed by Trump, a supporter of the U.S. coal industry who also questioned climate chang…

def recognition(sentence): persons = [] tokens = nltk.word_tokenize(sentence) tagged = nltk.pos_tag(tokens) entities = nltk.chunk.ne_chunk(tagged) for entity in entities.subtrees(): if entity.label() == "PERSON": name = [] for leaf in entity.leaves(): name.append(leaf[0]) persons.append(" ".join(name)) return persons

texts['persons'] = texts['text'].apply(recognition)

texts

textobject

7 others77.8%

personsobject

[]22.2%

['Joe Biden']11.1%

6 others66.7%

President Joe Biden's climate-related agenda, already under threat due to congressional failure to …

['Joe Biden']

The court's 6-3 conservative majority, suspicious of broad federal agency power, will weigh at oral…

['Clean Air Act']

An eventual ruling restricting EPA authority could hobble the administration's ability to curb the …

[]

"Could it be damaging? If it's an adverse decision, of course it could be," John Kerry, the Biden a…

['John Kerry']

The United States, trailing only China in greenhouse gas emissions, is a crucial player in global e…

[]

The case before the Supreme Court was brought by Republican-led states led by coal producer West Vi…

['Coal']

Democratic-led states and major power companies including Consolidated Edison Inc(ED.N), Exelon Cor…

['Exelon Corp', 'Biden']

The justices will review the U.S. Court of Appeals for the District of Columbia Circuit's 2021 ruli…

['Donald Trump']

The rule proposed by Trump, a supporter of the U.S. coal industry who also questioned climate chang…

['Trump', 'Barack Obama']