pip install --upgrade tensorflow
Requirement already satisfied: tensorflow in /shared-libs/python3.7/py/lib/python3.7/site-packages (2.4.1)
Collecting tensorflow
Downloading tensorflow-2.5.0-cp37-cp37m-manylinux2010_x86_64.whl (454.3 MB)
|████████████████████████████████| 454.3 MB 14 kB/s
Collecting tensorflow-estimator<2.6.0,>=2.5.0rc0
Downloading tensorflow_estimator-2.5.0-py2.py3-none-any.whl (462 kB)
|████████████████████████████████| 462 kB 69.9 MB/s
Collecting six~=1.15.0
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Requirement already satisfied: keras-preprocessing~=1.1.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.1.2)
Requirement already satisfied: wheel~=0.35 in /root/venv/lib/python3.7/site-packages (from tensorflow) (0.36.2)
Requirement already satisfied: astunparse~=1.6.3 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.6.3)
Collecting typing-extensions~=3.7.4
Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Requirement already satisfied: absl-py~=0.10 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (0.13.0)
Collecting grpcio~=1.34.0
Downloading grpcio-1.34.1-cp37-cp37m-manylinux2014_x86_64.whl (4.0 MB)
|████████████████████████████████| 4.0 MB 40.4 MB/s
Requirement already satisfied: flatbuffers~=1.12.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.12)
Requirement already satisfied: protobuf>=3.9.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (3.17.3)
Collecting gast==0.4.0
Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting keras-nightly~=2.5.0.dev
Downloading keras_nightly-2.5.0.dev2021032900-py2.py3-none-any.whl (1.2 MB)
|████████████████████████████████| 1.2 MB 53.8 MB/s
Requirement already satisfied: wrapt~=1.12.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.12.1)
Collecting h5py~=3.1.0
Downloading h5py-3.1.0-cp37-cp37m-manylinux1_x86_64.whl (4.0 MB)
|████████████████████████████████| 4.0 MB 50.7 MB/s
Requirement already satisfied: numpy~=1.19.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.19.5)
Requirement already satisfied: tensorboard~=2.5 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (2.5.0)
Requirement already satisfied: termcolor~=1.1.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.1.0)
Requirement already satisfied: google-pasta~=0.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (0.2.0)
Requirement already satisfied: opt-einsum~=3.3.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (3.3.0)
Collecting cached-property
Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Requirement already satisfied: markdown>=2.6.8 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow) (3.3.4)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow) (0.4.4)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow) (0.6.1)
Requirement already satisfied: setuptools>=41.0.0 in /root/venv/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow) (57.0.0)
Requirement already satisfied: requests<3,>=2.21.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow) (2.25.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow) (1.8.0)
Requirement already satisfied: werkzeug>=0.11.15 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow) (2.0.1)
Requirement already satisfied: google-auth<2,>=1.6.3 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow) (1.31.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow) (4.7.2)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow) (4.2.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.5->tensorflow) (1.3.0)
Requirement already satisfied: importlib-metadata in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from markdown>=2.6.8->tensorboard~=2.5->tensorflow) (4.5.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow) (0.4.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow) (1.26.5)
Requirement already satisfied: chardet<5,>=3.0.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow) (2.10)
Requirement already satisfied: oauthlib>=3.0.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.5->tensorflow) (3.1.1)
Requirement already satisfied: zipp>=0.5 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from importlib-metadata->markdown>=2.6.8->tensorboard~=2.5->tensorflow) (3.4.1)
Installing collected packages: typing-extensions, six, grpcio, cached-property, tensorflow-estimator, keras-nightly, h5py, gast, tensorflow
Attempting uninstall: typing-extensions
Found existing installation: typing-extensions 3.10.0.0
Uninstalling typing-extensions-3.10.0.0:
ERROR: Could not install packages due to an OSError: [Errno 30] Read-only file system: '/shared-libs/python3.7/py-core/lib/python3.7/site-packages/__pycache__/typing_extensions.cpython-37.pyc'
Note: you may need to restart the kernel to use updated packages.
from tensorflow import keras
data = keras.datasets.mnist.load_data(path="mnist.npz")
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 0s 0us/step
print(type(data))
<class 'tuple'>
print(data)
((array([[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
...,
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]]], dtype=uint8), array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)), (array([[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
...,
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]]], dtype=uint8), array([7, 2, 1, ..., 4, 5, 6], dtype=uint8)))
print(data[0])
(array([[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
...,
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]]], dtype=uint8), array([5, 0, 4, ..., 5, 6, 8], dtype=uint8))
train = data[0]
import matplotlib.pyplot as plt
(X_train, X_test), (y_train, y_test) = data
plt.imshow(X_train[50], cmap ='gray')
import numpy as np
print(np.shape(X_train))
(60000, 28, 28)
from sklearn.linear_model import LogisticRegression
import numpy as np
model = LogisticRegression()
data[0][0].shape
nbPixel = len(data[0][0][0])**2
X_train = data[0][0].reshape(data[0][0].shape[0], nbPixel)
print (X_train.shape)
X_test = data[1][0].reshape(data[1][0].shape[0], nbPixel)
print (X_test.shape)
(60000, 784)
(10000, 784)
y_train = data[0][1]
y_test = data[1][1]
model.fit(X_train, y_train)
/shared-libs/python3.7/py/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:765: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
print (model.score(X_train, y_train))
print (model.score(X_test, y_test))
0.9339166666666666
0.9255
from sklearn.metrics import confusion_matrix
print("__matrice train__")
print (confusion_matrix(model.predict(X_train), y_train))
print("\n__matrice de test__")
print(confusion_matrix(model.predict(X_test), y_test))
__matrice train__
[[5764 1 28 17 13 57 34 11 27 21]
[ 0 6584 47 23 22 18 9 20 93 22]
[ 15 32 5445 118 23 40 40 58 55 14]
[ 9 19 89 5582 9 161 0 28 122 70]
[ 14 6 61 6 5491 48 35 42 19 129]
[ 36 20 20 162 8 4793 58 8 136 33]
[ 33 3 56 14 48 82 5713 4 36 3]
[ 10 10 51 50 15 15 4 5896 18 138]
[ 38 55 140 119 42 160 21 20 5293 45]
[ 4 12 21 40 171 47 4 178 52 5474]]
__matrice de test__
[[ 963 0 3 4 1 11 9 1 8 9]
[ 0 1112 10 1 1 2 3 6 7 7]
[ 0 4 926 21 7 1 7 24 6 0]
[ 3 2 15 916 3 33 3 5 23 11]
[ 1 0 6 1 910 11 7 7 6 25]
[ 3 1 4 26 0 776 16 1 26 6]
[ 4 3 15 3 9 11 910 0 10 0]
[ 4 2 8 9 7 6 2 951 10 22]
[ 2 11 42 22 10 35 1 3 869 7]
[ 0 0 3 7 34 6 0 30 9 922]]
X_train = X_train / 255
X_test = X_test / 255
model.fit(X_train, y_train)
print (model.score(X_train, y_train))
print (model.score(X_test, y_test))
print("__matrice train__")
print (confusion_matrix(model.predict(X_train), y_train))
print("\n__matrice de test__")
print(confusion_matrix(model.predict(X_test), y_test))
/shared-libs/python3.7/py/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:765: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
0.9350666666666667
0.9258
__matrice train__
[[5768 1 25 17 11 48 27 8 26 21]
[ 1 6581 50 24 24 19 13 20 96 26]
[ 13 29 5444 118 24 38 38 62 56 14]
[ 8 19 91 5558 9 146 1 23 130 67]
[ 13 6 62 7 5506 45 35 41 24 130]
[ 40 26 24 184 7 4883 65 10 134 33]
[ 33 3 59 16 49 75 5710 4 37 3]
[ 8 13 57 48 19 18 5 5917 17 149]
[ 35 54 122 116 34 112 21 16 5276 45]
[ 4 10 24 43 159 37 3 164 55 5461]]
__matrice de test__
[[ 959 0 6 4 1 10 9 1 9 9]
[ 0 1111 9 1 1 2 3 8 11 8]
[ 0 4 926 18 7 3 8 24 8 0]
[ 3 2 16 917 3 34 2 5 23 11]
[ 1 0 9 1 914 7 7 7 7 24]
[ 7 2 4 22 0 783 14 1 25 6]
[ 5 3 13 4 10 14 912 0 12 0]
[ 4 2 6 11 4 6 2 950 7 19]
[ 1 11 39 25 10 29 1 3 861 7]
[ 0 0 4 7 32 4 0 29 11 925]]
X_train = X_train / 255
X_test = X_test / 255
model.fit(X_train, y_train)
print (model.score(X_train, y_train))
print (model.score(X_test, y_test))
print("__matrice train__")
print (confusion_matrix(model.predict(X_train), y_train))
print("\n__matrice de test__")
print(confusion_matrix(model.predict(X_test), y_test))
0.7637833333333334
0.7793
__matrice train__
[[5568 0 165 90 27 365 147 52 102 109]
[ 22 6667 729 450 342 1071 388 488 1079 416]
[ 32 22 4288 166 26 24 90 54 84 35]
[ 47 13 166 4967 4 1507 10 6 682 124]
[ 10 2 116 11 4793 137 44 72 51 463]
[ 3 3 0 16 1 1518 16 0 14 2]
[ 168 14 191 75 137 220 5193 4 85 12]
[ 28 10 166 135 93 225 13 5504 86 917]
[ 37 7 125 140 29 144 17 10 3502 44]
[ 8 4 12 81 390 210 0 75 166 3827]]
__matrice de test__
[[ 937 0 30 8 4 61 31 3 25 25]
[ 3 1123 156 54 62 144 51 85 128 50]
[ 3 4 721 24 2 2 12 17 12 8]
[ 7 3 30 868 0 263 1 1 110 12]
[ 0 0 19 1 821 25 10 10 11 85]
[ 1 0 0 2 0 265 6 0 2 0]
[ 22 4 24 5 32 38 845 2 20 3]
[ 3 0 27 22 9 50 1 896 29 122]
[ 4 1 24 20 2 19 1 2 619 6]
[ 0 0 1 6 50 25 0 12 18 698]]