!python --version
Python 3.7.12
pip --version
pip 20.1.1 from /root/venv/lib/python3.7/site-packages/pip (python 3.7)
Note: you may need to restart the kernel to use updated packages.
!pip install ipywidgets
#/root/venv/bin/python
!/root/venv/bin/python -m pip install --upgrade pip
!/usr/local/bin/python -m pip install ipywidgets
#/root/venv/bin/python
!/usr/local/bin/python -m jupyter nbextension enable --py widgetsnbextension
#/root/venv/bin/python
from ipywidgets import IntProgress
from flair.models.text_classification_model import TextClassifier
#classifier = TextClassifier.load('/datasets/gd/type_dis_trans_few/final-model.pt')
classifierTyp = TextClassifier.load('/datasets/gd/type_disease_trans_more/final-model.pt')
classifierMis = TextClassifier.load('/datasets/gd/pure_corpus_17k_slow_bert_latest/final-model.pt')
overall_score= 0
2022-02-04 11:28:20,718 loading file /datasets/gd/type_disease_trans_more/final-model.pt
IPyWidgets are not supported
IPyWidgets are not supported
IPyWidgets are not supported
IPyWidgets are not supported
2022-02-04 11:28:30,021 loading file /datasets/gd/pure_corpus_17k_slow_bert_latest/final-model.pt
!pip install -U spacy
!python -m spacy validate
Requirement already satisfied: spacy in /shared-libs/python3.7/py/lib/python3.7/site-packages (3.2.1)
Requirement already satisfied: setuptools in /root/venv/lib/python3.7/site-packages (from spacy) (47.1.0)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (0.7.5)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (3.0.6)
Requirement already satisfied: srsly<3.0.0,>=2.4.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (2.4.2)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (2.26.0)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (1.0.1)
Requirement already satisfied: thinc<8.1.0,>=8.0.12 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (8.0.13)
Requirement already satisfied: jinja2 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from spacy) (3.0.3)
Collecting typing-extensions<4.0.0.0,>=3.7.4
Downloading typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Requirement already satisfied: pathy>=0.3.5 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (0.6.1)
Requirement already satisfied: typer<0.5.0,>=0.3.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (0.4.0)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (2.0.6)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (1.8.2)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (4.62.3)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (3.0.8)
Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (0.9.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (3.3.0)
Requirement already satisfied: numpy>=1.15.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (1.19.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (1.0.6)
Requirement already satisfied: packaging>=20.0 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from spacy) (21.3)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from spacy) (2.0.6)
Requirement already satisfied: zipp>=0.5 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from catalogue<2.1.0,>=2.0.6->spacy) (3.6.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from packaging>=20.0->spacy) (3.0.6)
Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from pathy>=0.3.5->spacy) (5.2.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2021.10.8)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from typer<0.5.0,>=0.3.0->spacy) (8.0.3)
Requirement already satisfied: MarkupSafe>=2.0 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from jinja2->spacy) (2.0.1)
Requirement already satisfied: importlib-metadata in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from click<9.0.0,>=7.1.1->typer<0.5.0,>=0.3.0->spacy) (4.8.2)
Installing collected packages: typing-extensions
Attempting uninstall: typing-extensions
Found existing installation: typing_extensions 4.0.1
Not uninstalling typing-extensions at /shared-libs/python3.7/py-core/lib/python3.7/site-packages, outside environment /root/venv
Can't uninstall 'typing_extensions'. No files were found to uninstall.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
konoha 4.6.5 requires importlib-metadata<4.0.0,>=3.7.0, but you have importlib-metadata 4.8.2 which is incompatible.
tensorflow 2.4.1 requires six~=1.15.0, but you have six 1.16.0 which is incompatible.
tensorflow 2.4.1 requires typing-extensions~=3.7.4, but you have typing-extensions 3.10.0.2 which is incompatible.
Successfully installed typing-extensions-3.10.0.2
2022-02-04 11:39:47.498256: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-02-04 11:39:47.498316: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
✔ Loaded compatibility table
================= Installed pipeline packages (spaCy v3.2.1) =================
ℹ spaCy installation:
/shared-libs/python3.7/py/lib/python3.7/site-packages/spacy
No pipeline packages found in your current environment.
display(HTML(highlighted_text))
df_coeff["normalized_score"].mean()
display(HTML(highlighted_text))
df_coeff["normalized_score"].mean()
anvil.server.wait_forever()
/shared-libs/python3.7/py-core/lib/python3.7/site-packages/ipykernel_launcher.py:96: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
/shared-libs/python3.7/py-core/lib/python3.7/site-packages/ipykernel_launcher.py:113: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
/shared-libs/python3.7/py-core/lib/python3.7/site-packages/ipykernel_launcher.py:117: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
Hi im avg_score -0.9999788999557495
Hi im avg_score 0.9993174076080322
Hi im avg_score -0.9997898936271667
Hi im avg_score -0.9833953380584717
Hi im avg_score 0.5455038547515869
Hi im avg_score 0.9986968636512756
Hi im avg_score 0.9999421834945679
Hi im avg_score 0.9989527463912964
Hi im avg_score -0.9999673366546631
Hi im avg_score 0.9930728077888489
Hi im avg_score -0.9985535740852356
Hi im avg_score -0.999835729598999
Hi im avg_score -0.9992017149925232
normalized Multi records 0 = 1.055002212524414e-05
normalized Multi records 1 = 0.9996587038040161
sentence \
0 There's nothing you can do to prevent heart disease.
1 Smokers are more likely to have heart disease than nonsmokers.
2 Some risk factors for heart disease can't be changed.
3 You have to exercise at least 1 hour a day to reduce your risk for heart disease.
4 Drinking 3 to 4 alcoholic drinks each day can reduce your risk for heart disease.
5 High blood pressure can put your heart at risk.
6 An average of 100,000 Americans die from heart disease every year.
7 Someone who has had a heart attack is at increased risk of having another.
8 You can't exercise if you have heart disease.
9 Being overweight increases your risk for heart disease.
10 Emotions don't affect your risk for heart disease.
11 Your diet doesn't affect your risk for heart disease.
12 No tests can diagnose coronary heart disease.
avg_score label_type score_type \
0 -0.999979 [heart disease causes]. 0.879007
1 0.999317 [Coronary artery disease]. 0.723772
2 -0.999790 [heart disease causes]. 0.612359
3 -0.983395 [Coronary artery disease]. 0.950520
4 0.545504 [Coronary artery disease]. 0.964518
5 0.998697 [heart disease causes]. 0.669329
6 0.999942 [Coronary artery disease treatment]. 0.362643
7 0.998953 [heart attack]. 0.996314
8 -0.999967 [Coronary artery disease]. 0.863569
9 0.993073 [heart disease causes]. 0.653473
10 -0.998554 [Coronary artery disease]. 0.940347
11 -0.999836 [heart disease causes]. 0.983456
12 -0.999202 [Coronary artery disease]. 0.485939
normalized_score normalized_full
0 0.000011 0.000011
1 0.999659 0.999659
2 0.000105 0.000105
3 0.008302 0.008302
4 0.772752 0.772752
5 0.999348 0.999348
6 0.999971 0.999971
7 0.999476 0.999476
8 0.000016 0.000016
9 0.996536 0.996536
10 0.000723 0.000723
11 0.000082 0.000082
12 0.000399 0.000399