Part A
import pandas as pd
import numpy as np
!pip install bs4==0.0.1
Collecting bs4==0.0.1
Downloading bs4-0.0.1.tar.gz (1.1 kB)
Collecting beautifulsoup4
Downloading beautifulsoup4-4.10.0-py3-none-any.whl (97 kB)
|████████████████████████████████| 97 kB 6.7 MB/s
Collecting soupsieve>1.2
Downloading soupsieve-2.3.1-py3-none-any.whl (37 kB)
Building wheels for collected packages: bs4
Building wheel for bs4 (setup.py) ... done
Created wheel for bs4: filename=bs4-0.0.1-py3-none-any.whl size=1272 sha256=6638c89aadf1791c683068f50fde1c7aeb41cfa22fb126cf3278228f51d35b28
Stored in directory: /root/.cache/pip/wheels/0a/9e/ba/20e5bbc1afef3a491f0b3bb74d508f99403aabe76eda2167ca
Successfully built bs4
Installing collected packages: soupsieve, beautifulsoup4, bs4
Successfully installed beautifulsoup4-4.10.0 bs4-0.0.1 soupsieve-2.3.1
WARNING: You are using pip version 20.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/root/venv/bin/python -m pip install --upgrade pip' command.
from bs4 import BeautifulSoup
train = pd.read_csv('stack_stats_2020_train.csv')
test = pd.read_csv('stack_stats_2020_test.csv')
stack_stats = pd.concat([train, test])
stack_stats.head()
#create a function that turns html text into plain text
def plain_text(table, column):
plain_text = []
for index, row in column