!pip install tensorflow
Requirement already satisfied: tensorflow in /shared-libs/python3.7/py/lib/python3.7/site-packages (2.4.1)
Requirement already satisfied: termcolor~=1.1.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.1.0)
Requirement already satisfied: flatbuffers~=1.12.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.12)
Requirement already satisfied: keras-preprocessing~=1.1.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.1.2)
Requirement already satisfied: numpy~=1.19.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.19.5)
Requirement already satisfied: gast==0.3.3 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (0.3.3)
Requirement already satisfied: tensorboard~=2.4 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (2.5.0)
Requirement already satisfied: absl-py~=0.10 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (0.13.0)
Requirement already satisfied: grpcio~=1.32.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.32.0)
Requirement already satisfied: h5py~=2.10.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (2.10.0)
Requirement already satisfied: wrapt~=1.12.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.12.1)
Requirement already satisfied: protobuf>=3.9.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (3.17.3)
Requirement already satisfied: astunparse~=1.6.3 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (1.6.3)
Requirement already satisfied: opt-einsum~=3.3.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (3.3.0)
Collecting six~=1.15.0
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Requirement already satisfied: google-pasta~=0.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (0.2.0)
Requirement already satisfied: wheel~=0.35 in /root/venv/lib/python3.7/site-packages (from tensorflow) (0.36.2)
Collecting typing-extensions~=3.7.4
Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Requirement already satisfied: tensorflow-estimator<2.5.0,>=2.4.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorflow) (2.4.0)
Requirement already satisfied: requests<3,>=2.21.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow) (2.25.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow) (1.8.0)
Requirement already satisfied: markdown>=2.6.8 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow) (3.3.4)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow) (0.4.4)
Requirement already satisfied: setuptools>=41.0.0 in /root/venv/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow) (57.0.0)
Requirement already satisfied: google-auth<2,>=1.6.3 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow) (1.31.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow) (0.6.1)
Requirement already satisfied: werkzeug>=0.11.15 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from tensorboard~=2.4->tensorflow) (2.0.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow) (4.7.2)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow) (4.2.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.4->tensorflow) (1.3.0)
Requirement already satisfied: importlib-metadata in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from markdown>=2.6.8->tensorboard~=2.4->tensorflow) (4.5.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.4->tensorflow) (0.4.8)
Requirement already satisfied: idna<3,>=2.5 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow) (2020.12.5)
Requirement already satisfied: chardet<5,>=3.0.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow) (3.0.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.4->tensorflow) (1.26.5)
Requirement already satisfied: oauthlib>=3.0.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.4->tensorflow) (3.1.1)
Requirement already satisfied: zipp>=0.5 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from importlib-metadata->markdown>=2.6.8->tensorboard~=2.4->tensorflow) (3.4.1)
Installing collected packages: typing-extensions, six
Attempting uninstall: typing-extensions
Found existing installation: typing-extensions 3.10.0.0
Not uninstalling typing-extensions at /shared-libs/python3.7/py-core/lib/python3.7/site-packages, outside environment /root/venv
Can't uninstall 'typing-extensions'. No files were found to uninstall.
Attempting uninstall: six
Found existing installation: six 1.16.0
Not uninstalling six at /shared-libs/python3.7/py-core/lib/python3.7/site-packages, outside environment /root/venv
Can't uninstall 'six'. No files were found to uninstall.
Successfully installed six-1.15.0 typing-extensions-3.7.4.3
from tensorflow.keras.datasets.mnist import load_data
data = load_data(path="mnist.npz")
print(type(data))
<class 'tuple'>
print(type(data))
<class 'tuple'>
data[0]
print(data[0])
(array([[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
...,
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]],
[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]]], dtype=uint8), array([5, 0, 4, ..., 5, 6, 8], dtype=uint8))
import matplotlib.pyplot as plt
(x_train, y_train), (x_test, y_test) = data
plt.imshow(x_train[0])
plt.imshow(x_train[1])
plt.imshow(x_train[2])
print(x_train.shape)
print(x_test.shape)
print(y_train.shape)
print(y_test.shape)
(60000, 28, 28)
(10000, 28, 28)
(60000,)
(10000,)
from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
x_train = x_train.reshape(x_train.shape[0], 784)
y_train = y_train.reshape(y_train.shape[0])
y_test = y_test.reshape(y_test.shape[0])
x_test = x_test.reshape(x_test.shape[0], 784)
model.fit(x_train, y_train)
/shared-libs/python3.7/py/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:765: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
print(model.score(x_train, y_train))
print(model.score(x_test, y_test))
0.9339166666666666
0.9255
from sklearn.metrics import confusion_matrix, plot_confusion_matrix
plot_confusion_matrix(model,x_test,y_test)
x_test = x_test/255
x_train = x_train/255
model.fit(x_train,y_train)
print("score du train")
print(model.score(x_train, y_train))
print("score du test")
print(model.score(x_test, y_test))
y_preds = model.predict(x_test)
print("matrice de confusion classique")
conf_mat = confusion_matrix(y_test, y_preds)
print(conf_mat)
model.fit(x_train,y_train)
print("score normalisé du train",model.score(x_train, y_train))
print("score normalisé du test",model.score(x_test, y_test))
y_preds = model.predict(x_test)
conf_mat = confusion_matrix(y_test, y_preds)
print("matrice de confusion",conf_mat)
score du train
0.7637833333333334
score du test
0.7793
matrice de confusion classique
[[ 937 3 3 7 0 1 22 3 4 0]
[ 0 1123 4 3 0 0 4 0 1 0]
[ 30 156 721 30 19 0 24 27 24 1]
[ 8 54 24 868 1 2 5 22 20 6]
[ 4 62 2 0 821 0 32 9 2 50]
[ 61 144 2 263 25 265 38 50 19 25]
[ 31 51 12 1 10 6 845 1 1 0]
[ 3 85 17 1 10 0 2 896 2 12]
[ 25 128 12 110 11 2 20 29 619 18]
[ 25 50 8 12 85 0 3 122 6 698]]
score normalisé du train 0.7637833333333334
score normalisé du test 0.7793
matrice de confusion [[ 937 3 3 7 0 1 22 3 4 0]
[ 0 1123 4 3 0 0 4 0 1 0]
[ 30 156 721 30 19 0 24 27 24 1]
[ 8 54 24 868 1 2 5 22 20 6]
[ 4 62 2 0 821 0 32 9 2 50]
[ 61 144 2 263 25 265 38 50 19 25]
[ 31 51 12 1 10 6 845 1 1 0]
[ 3 85 17 1 10 0 2 896 2 12]
[ 25 128 12 110 11 2 20 29 619 18]
[ 25 50 8 12 85 0 3 122 6 698]]
from tensorflow import keras
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix, plot_confusion_matrix
from sklearn.linear_model import LogisticRegression
(x_train, y_train), (x_test, y_test) = keras.datasets.cifar10.load_data()
plt.imshow(x_train[0])
plt.figure()
plt.imshow(x_train[1])
plt.figure()
plt.imshow(x_train[2])
plt.figure()
print(x_train.shape)
print(x_test.shape)
print(y_train.shape)
print(y_test.shape)
x_train = x_train.reshape(x_train.shape[0], 3072)
x_test = x_test.reshape(x_test.shape[0], 3072)
y_train = y_train.reshape(y_train.shape[0])
y_test = y_test.reshape(y_test.shape[0])
model = LogisticRegression()
model.fit(x_train,y_train)
print("score du train")
print(model.score(x_train, y_train))
print("score du test")
print(model.score(x_test, y_test))
print("matrice de confusion en couleur")
plot_confusion_matrix(model,x_test,y_test)
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz
170500096/170498071 [==============================] - 3s 0us/step
(50000, 32, 32, 3)
(10000, 32, 32, 3)
(50000, 1)
(10000, 1)
/shared-libs/python3.7/py/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:765: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
score du train
0.4258
score du test
0.4024
matrice de confusion en couleur