Machine Learning Analysis on Logistic Regression, Decision Tree, and Random Forests Using the Mobile Price Classification Dataset
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.metrics import accuracy_score,confusion_matrix
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import f_classif
from sklearn.feature_selection import chi2
Introduction
Background of the Study
Statement of the Problem
Significance of the Study
Conducting the study provides several benefits for the smartphone market. Mainly, this revolves on the primary objective of the study which is determining the appropriate price range of a smartphone. Consumers that intend to purchase or inquire about the prices of a product can definitely benefit from the study. This is because the project develops a system using various machine learning models that identifies the appropriate price class or category of a phone given its specifications. Therefore, this can aid towards considering the affordability and cost-efficiency of products. Given these capabilities, the study also benefits the manufacturers. This is because it can aid in making informed decisions towards placing a smart phone in the market. Specifically, it allows them to estimate a competitive price for a product they are developing.
Mobile Price Classification Dataset
df = pd.read_csv('train.csv')
df.head()
The dataset is entitled Mobile Price Classification which was created by Abhishek Sharma. This was created in 2018, and was made publicly available in Kaggle. It contains information about smartphones and their corresponding specifications. All of which are smartphones that were released during 2017 and the years before it. Each row represents an entry of a single smartphone. There are 3000 samples that were included in the dataset. This is further divided into 2 sets which are the training set, containing 2000 samples, and the test set, consisting of 1000 samples. However, only the training set is considered for the methodologies since the test set does not contain the labels in each row. In addition to this, there are 21 columns that indicate the 20 features of a particular product, and the corresponding price range.
Features
Labels
Mobile Price Classification Dataset Class Distribution
sns.countplot(df['price_range'])
plt.xlabel('Price Range')
plt.title('Distribution of the Smartphones in Different Price Ranges')
plt.show()
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
Methodology
df.shape
print(df.dtypes)
battery_power int64
blue int64
clock_speed float64
dual_sim int64
fc int64
four_g int64
int_memory int64
m_dep float64
mobile_wt int64
n_cores int64
pc int64
px_height int64
px_width int64
ram int64
sc_h int64
sc_w int64
talk_time int64
three_g int64
touch_screen int64
wifi int64
price_range int64
dtype: object
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2000 entries, 0 to 1999
Data columns (total 21 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 battery_power 2000 non-null int64
1 blue 2000 non-null int64
2 clock_speed 2000 non-null float64
3 dual_sim 2000 non-null int64
4 fc 2000 non-null int64
5 four_g 2000 non-null int64
6 int_memory 2000 non-null int64
7 m_dep 2000 non-null float64
8 mobile_wt 2000 non-null int64
9 n_cores 2000 non-null int64
10 pc 2000 non-null int64
11 px_height 2000 non-null int64
12 px_width 2000 non-null int64
13 ram 2000 non-null int64
14 sc_h 2000 non-null int64
15 sc_w 2000 non-null int64
16 talk_time 2000 non-null int64
17 three_g 2000 non-null int64
18 touch_screen 2000 non-null int64
19 wifi 2000 non-null int64
20 price_range 2000 non-null int64
dtypes: float64(2), int64(19)
memory usage: 328.2 KB
df.describe()
df['price_range'].unique()
df.isnull().sum()
X = df.drop('price_range', axis=1) #independent columns
y = df['price_range'] #target column i.e price range
df.columns
categorical_col = ['blue','dual_sim','four_g','three_g',
'touch_screen','wifi']
fig, ax = plt.subplots(2,3)
fig.set_size_inches(10, 5)
i = 0
for feature in categorical_col:
if i < 3:
sns.countplot(df[feature], ax=ax[0, i])
else:
sns.countplot(df[feature], ax=ax[1, i-4])
i = i+1
plt.title('Value Count of Categorical Variables')
plt.tight_layout()
plt.show()
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
fig, ax = plt.subplots(2,7)
fig.set_size_inches(18.5, 5)
i = 0
for feature in df.drop(df[categorical_col],axis=1):
if i < 7:
sns.histplot(df[feature], ax=ax[0, i])
else:
sns.histplot(df[feature], ax=ax[1, i-8])
i = i+1
plt.title('Distribution to Values of Numerical Variables')
plt.tight_layout()
plt.show()
pplot = sns.pointplot(y='ram',x='price_range',data=df)
pplot.set_title("Correlation of Price Range and RAM")
box_plot = sns.boxplot(x='price_range',y='battery_power',data=df)
box_plot.set_title("Correlation of Price Range and Battery Power")
box_plot = sns.boxplot(x='price_range',y='px_height',data=df)
box_plot.set_title("Correlation of Price Range and Pixel Height")
box_plot = sns.boxplot(x='price_range',y='px_width',data=df)
box_plot.set_title("Correlation of Price Range and Pixel Width")
Exploring the other features
plt.figure(figsize=(10,6))
df['fc'].hist(alpha=0.5,color='blue',label='Front camera')
df['pc'].hist(alpha=0.5,color='red',label='Primary camera')
plt.legend()
plt.xlabel('MegaPixels')
plt.title("Megapixels of Front Camera and Primary Camera to All Phones")
labels = ["3G-supported",'Not supported']
values = df['three_g'].value_counts().values
fig1, ax1 = plt.subplots()
ax1.pie(values, labels=labels, autopct='%1.1f%%',startangle=90)
plt.title("Pie Chart on Percentage of 3G-Supported and Not 3G-supported Phones")
plt.show()
labels = ["4G-supported",'Not supported']
values = df['four_g'].value_counts().values
fig1, ax1 = plt.subplots()
ax1.pie(values, labels=labels, autopct='%1.1f%%',startangle=90)
plt.title("Pie Chart on Percentage of 4G-Supported and not 4G-Supported Phones")
plt.show()
One approach to get the most important features is with the use of an ensemble learning method fundamentally based on decision tree called ExtraTreesClassifier. This model has a property called feature_importances_ that states the score of each feature regarding its correlation to the target variable. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature, also known as the Gini importance.
# Initializing the ExtraTreesClassifier Model
model = ExtraTreesClassifier()
# Training the Model
model.fit(X,y)
# Getting the most important features and plotting it on a bar graph
important = pd.Series(model.feature_importances_, index=X.columns)
important.sort_values().plot(kind='barh')
plt.title("Most Important Features that are Correlated with Price Range")
plt.show()
In general, the proponents also wants to explore the correlation of the different features, aside from the target feature, in order to know which variable are similar to each other. With this, the use of Seaborn Correlation Heatmap is necessary since they show at a glance which variables are correlated, to what degree, and in which direction it correlates.
# compute the correlation matrix
corr = df.drop('price_range', axis=1).corr()
# Generate a mask for the upper triangle
mask = np.triu(np.ones_like(corr, dtype=np.bool))
# set the figure size
fig = plt.figure(figsize=(15,12))
# custom diverging colormap
cmap = sns.diverging_palette(220, 20, as_cmap=True)
#Draw the heatmap
mob_map = sns.heatmap(corr, mask=mask, cmap=cmap, vmax=1, vmin=-1, center=0,
square=True, linewidths=.5, cbar_kws={"shrink": .5})
mob_map.set_title("Correlation of Each Variables in the Mobile Price Classification Dataset")
selected_features = ['ram','battery_power','px_height','px_width', 'int_memory', 'mobile_wt', 'sc_h', 'talk_time', 'sc_w', 'clock_speed', 'm_dep', 'pc', 'n_cores', 'fc']
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, classification_report
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
from sklearn import tree
RANDOM_STATE = 42
# split data into training and testing data sets
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=RANDOM_STATE, test_size=0.3)
# select the best feature among them
X_train_sf = X_train[selected_features]
X_test_sf = X_test[selected_features]
lr_model = LogisticRegression(random_state=RANDOM_STATE, multi_class='multinomial')
lr_model.fit(X_train, y_train)
/shared-libs/python3.7/py/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:765: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
lr_preds = lr_model.predict(X_test)
accuracy_score(y_test, lr_preds)
In order to visualize the performance measures of the logistic regression with regards to all the features, the classification report is printed to show the precision, recall, f1-score, and support for every target variable by comparing the predictions of the model to the actual target variables. The test set yielded 63.5% accuracy for the model.
print(classification_report(y_test, lr_preds, digits=4))
precision recall f1-score support
0 0.8028 0.7550 0.7782 151
1 0.5175 0.5068 0.5121 146
2 0.5130 0.5338 0.5232 148
3 0.7081 0.7355 0.7215 155
accuracy 0.6350 600
macro avg 0.6353 0.6328 0.6337 600
weighted avg 0.6374 0.6350 0.6359 600
heat_map = sns.heatmap(confusion_matrix(y_test, lr_preds), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Linear Regression for All Features")
heat_map.set(xlabel='predicted', ylabel='actual')
# lr_model_sf = LogisticRegression(max_iter=200, solver='newton-cg', multi_class='multinomial', random_state=RANDOM_STATE)
lr_model_sf = LogisticRegression(multi_class='multinomial', random_state=RANDOM_STATE)
lr_model_sf.fit(X_train_sf, y_train)
/shared-libs/python3.7/py/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:765: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
lr_preds_sf = lr_model_sf.predict(X_test_sf)
accuracy_score(y_test, lr_preds_sf)
In order to visualize the performance measures of the logistic regression with regards to the selected features, the classification report is printed to show the precision, recall, f1-score, and support for every target variable by comparing the predictions of the model to the actual target variables.
print(classification_report(y_test, lr_preds_sf, digits=4))
precision recall f1-score support
0 0.8071 0.7483 0.7766 151
1 0.5139 0.5068 0.5103 146
2 0.5065 0.5270 0.5166 148
3 0.6975 0.7290 0.7129 155
accuracy 0.6300 600
macro avg 0.6313 0.6278 0.6291 600
weighted avg 0.6333 0.6300 0.6312 600
heat_map = sns.heatmap(confusion_matrix(y_test, lr_preds_sf), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Linear Regression for Selected Features")
heat_map.set(xlabel='predicted', ylabel='actual')
lr_params = {
'tol': [1e-2, 1e-3, 1e-4, 1e-5],
'max_iter': [100, 200, 300, 400, 500, 600, 700, 1000],
'solver': ['newton-cg', 'lbfgs', 'liblinear'],
'multi_class': ['multinomial']
}
Snlr_gs_sf = GridSearchCV(LogisticRegression(random_state=RANDOM_STATE), lr_params, cv=5, n_jobs=-1)
Snlr_gs_sf.fit(X_train_sf, y_train)
/shared-libs/python3.7/py/lib/python3.7/site-packages/sklearn/model_selection/_search.py:925: UserWarning: One or more of the test scores are non-finite: [0.95857143 0.95857143 0.95857143 0.95857143 0.63642857 0.63642857
0.63642857 0.63642857 nan nan nan nan
0.96357143 0.96357143 0.96357143 0.96357143 0.65714286 0.65714286
0.65714286 0.65714286 nan nan nan nan
0.96642857 0.96642857 0.96642857 0.96642857 0.67857143 0.67857143
0.67857143 0.67857143 nan nan nan nan
0.96428571 0.96428571 0.96428571 0.96428571 0.68928571 0.68928571
0.68928571 0.68928571 nan nan nan nan
0.96428571 0.96428571 0.96428571 0.96428571 0.68857143 0.68857143
0.68857143 0.68857143 nan nan nan nan
0.96285714 0.96285714 0.96285714 0.96285714 0.70071429 0.70071429
0.70071429 0.70071429 nan nan nan nan
0.96285714 0.96285714 0.96285714 0.96285714 0.69285714 0.69285714
0.69285714 0.69285714 nan nan nan nan
0.96285714 0.96285714 0.96285714 0.96285714 0.69428571 0.69428571
0.69428571 0.69428571 nan nan nan nan]
category=UserWarning
/shared-libs/python3.7/py/lib/python3.7/site-packages/sklearn/utils/optimize.py:203: ConvergenceWarning: newton-cg failed to converge. Increase the number of iterations.
"number of iterations.", ConvergenceWarning)
Snlr_gs_sf_acc = Snlr_gs_sf .best_score_
Snlr_gs_sf_acc
lr_gs_sf_results = pd.DataFrame(Snlr_gs_sf.cv_results_)
Snlr_gs_sf.best_params_
lr_gs_preds_sf = Snlr_gs_sf.best_estimator_.predict(X_test_sf)
lr_acc = accuracy_score(y_test, lr_gs_preds_sf)
lr_acc
print(classification_report(y_test, lr_gs_preds_sf, digits=4))
precision recall f1-score support
0 0.9933 0.9801 0.9867 151
1 0.9416 0.9932 0.9667 146
2 0.9928 0.9324 0.9617 148
3 0.9747 0.9935 0.9840 155
accuracy 0.9750 600
macro avg 0.9756 0.9748 0.9748 600
weighted avg 0.9758 0.9750 0.9750 600
heat_map = sns.heatmap(confusion_matrix(y_test, lr_gs_preds_sf), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Hyperparameter Tuned Linear Regression for Selected Features")
heat_map.set(xlabel='predicted', ylabel='actual')
First, the researchers will run a baseline model with default settings on the initial dataset.
# setup and train the model
dtc_model = DecisionTreeClassifier(random_state=RANDOM_STATE)
dtc_model.fit(X_train, y_train)
# test the model
dt_predictions = dtc_model.predict(X_test)
accuracy_score(y_test, dt_predictions)
From the results, it can be seen that the model scores an average of 81.83%. This score will serve as a point of comparison to our model with selected features.
print(classification_report(y_test, dt_predictions, digits=4))
precision recall f1-score support
0 0.9097 0.8675 0.8881 151
1 0.7421 0.8082 0.7738 146
2 0.7517 0.7365 0.7440 148
3 0.8750 0.8581 0.8664 155
accuracy 0.8183 600
macro avg 0.8196 0.8176 0.8181 600
weighted avg 0.8210 0.8183 0.8192 600
# print the confusion matrix
heat_map = sns.heatmap(confusion_matrix(y_test, dt_predictions), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Decision Tree for All Features")
heat_map.set(xlabel='predicted', ylabel='actual')
tree.export_graphviz(dtc_model,out_file='dtc_model.dot')
The researches will now use the Decision Tree Classifier with only the selected features.
dtc_model_sf = DecisionTreeClassifier(random_state=RANDOM_STATE)
dtc_model_sf.fit(X_train_sf, y_train)
dt_predictions_sf = dtc_model_sf.predict(X_test_sf)
accuracy_score(y_test, dt_predictions_sf)
print(classification_report(y_test, dt_predictions_sf, digits=4))
precision recall f1-score support
0 0.89 0.87 0.88 151
1 0.74 0.78 0.76 146
2 0.73 0.70 0.72 148
3 0.86 0.85 0.85 155
accuracy 0.80 600
macro avg 0.80 0.80 0.80 600
weighted avg 0.80 0.80 0.80 600
heat_map = sns.heatmap(confusion_matrix(y_test, dt_predictions_sf), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Decision Tree for Selected Features")
heat_map.set(xlabel='predicted', ylabel='actual')
tree.export_graphviz(dtc_model_sf,out_file='dtc_model_sf.dot')
dt_params = {
'splitter': ['best', 'random'],
'min_samples_split' : [2, 5, 10, 100, 500, 1000],
'min_samples_leaf' : [1, 2, 4, 8, 16],
'max_features': ['sqrt', 'log2', None]
}
dt_grid_search_sf = GridSearchCV(DecisionTreeClassifier(random_state=RANDOM_STATE), dt_params, n_jobs=-1, cv=5, verbose=0)
dt_grid_search_sf.fit(X_train_sf, y_train)
dt_acc = dt_grid_search_sf.best_score_
dt_acc
dt_grid_search_results = pd.DataFrame(dt_grid_search_sf.cv_results_).fillna(value='None')
dt_grid_search_sf.best_params_
dt_grid_preds_sf = dt_grid_search_sf.best_estimator_.predict(X_test_sf)
dt_grid_acc = accuracy_score(y_test, dt_grid_preds_sf)
dt_grid_acc
print(classification_report(y_test, dt_grid_preds_sf))
precision recall f1-score support
0 0.93 0.88 0.90 151
1 0.74 0.79 0.76 146
2 0.74 0.76 0.75 148
3 0.91 0.88 0.89 155
accuracy 0.83 600
macro avg 0.83 0.83 0.83 600
weighted avg 0.83 0.83 0.83 600
heat_map = sns.heatmap(confusion_matrix(y_test, dt_grid_preds_sf), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Hyperparameter Tuned Decision Tree for Selected Features")
heat_map.set(xlabel='predicted', ylabel='actual')
tree.export_graphviz(dt_grid_search_sf.best_estimator_,out_file='dt_grid_search_sf.dot')
# Set up the model and fit the initial training data
rfc_model = RandomForestClassifier(random_state=RANDOM_STATE)
rfc_model.fit(X_train, y_train)
# Print the classification report
predictions = rfc_model.predict(X_test)
accuracy_score(y_test, predictions)
print(classification_report(y_test, predictions, digits=4))
precision recall f1-score support
0 0.9351 0.9536 0.9443 151
1 0.8151 0.8151 0.8151 146
2 0.7891 0.7838 0.7864 148
3 0.9216 0.9097 0.9156 155
accuracy 0.8667 600
macro avg 0.8652 0.8655 0.8653 600
weighted avg 0.8664 0.8667 0.8665 600
heat_map = sns.heatmap(confusion_matrix(y_test, predictions), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Random Forests for All Features")
heat_map.set(xlabel='predicted', ylabel='actual')
In this section, the Random Forest model is trained using only the selected features.
rfc_model_sf = RandomForestClassifier(random_state=RANDOM_STATE)
rfc_model_sf.fit(X_train_sf, y_train)
predictions_sf = rfc_model_sf.predict(X_test_sf)
accuracy_score(y_test, predictions_sf)
print(classification_report(y_test, predictions_sf, digits=4))
precision recall f1-score support
0 0.9477 0.9603 0.9539 151
1 0.8366 0.8767 0.8562 146
2 0.8112 0.7838 0.7973 148
3 0.9139 0.8903 0.9020 155
accuracy 0.8783 600
macro avg 0.8774 0.8778 0.8773 600
weighted avg 0.8783 0.8783 0.8781 600
heat_map = sns.heatmap(confusion_matrix(y_test, predictions_sf), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Random Forests for Selected Features")
heat_map.set(xlabel='predicted', ylabel='actual')
rf_params = {
'n_estimators': [100, 250, 500],
'min_samples_split' : [5, 10, 100, 1000],
'min_samples_leaf' : [2, 4, 8, 16],
'max_features': ['sqrt', 'log2', None],
'bootstrap' : [True, False]
}
rf_grid_search_sf = GridSearchCV(RandomForestClassifier(random_state=RANDOM_STATE), rf_params, n_jobs=-1, cv=5, verbose=0)
rf_grid_search_sf.fit(X_train_sf, y_train)
rf_acc = rf_grid_search_sf.best_score_
rf_acc
rf_grid_search_sf.best_params_
rf_grid_preds_sf = rf_grid_search_sf.best_estimator_.predict(X_test_sf)
rf_grid_acc = accuracy_score(y_test, rf_grid_preds_sf)
rf_grid_acc
rf_hpt_df = pd.DataFrame(rf_grid_search_sf.cv_results_).fillna(value='None')
In order to visualize the performance measures of the Random Forest Classifier with regards to selected features, the classification report is printed to show the precision, recall, f1 score, and support for every target variable by comparing the predictions of the model to the actual target variables. After going through hyperparameter tuning, the model did improve. However, this is not as significant compared to other models. Specifically, there is only a 0.34% improvement from the model that contained the default parameters to the model which used the best set of parameters from the grid search.
print(classification_report(rf_grid_preds_sf, y_test, digits=4))
precision recall f1-score support
0 0.9139 0.9388 0.9262 147
1 0.8904 0.8387 0.8638 155
2 0.8311 0.8367 0.8339 147
3 0.8903 0.9139 0.9020 151
accuracy 0.8817 600
macro avg 0.8814 0.8820 0.8815 600
weighted avg 0.8816 0.8817 0.8814 600
heat_map = sns.heatmap(confusion_matrix(y_test, rf_grid_preds_sf), annot=True, fmt='d')
heat_map.set_title("Confusion Matrix on Hyperparameter Tuned Random Forests for Selected Features")
heat_map.set(xlabel='predicted', ylabel='actual')
Results and Analysis
lr_gs_results_fill_none = lr_gs_sf_results.fillna(value=0)
lr_hp = ['param_max_iter', 'param_solver', 'param_tol']
lr_hpt_selected = lr_gs_results_fill_none[['param_max_iter', 'param_multi_class', 'param_solver', 'param_tol', 'mean_test_score']]
fig, axs = plt.subplots(1, len(lr_hp), figsize=(20,5), sharey='all')
for i, hp in enumerate(lr_hp):
lr_hpt_selected.groupby(hp)['mean_test_score'].mean().plot.bar(ax=axs[i], ylim=(0, 1.0))
axs[i].tick_params(labelrotation=0)
fig.suptitle('Average mean_test_score for Hyperparameter Settings for Logistic Regression')
plt.show()
Decision Tree
dt_hp = ['param_max_features', 'param_min_samples_leaf', 'param_min_samples_split', 'param_splitter']
dt_hpt_selected = dt_grid_search_results[['param_max_features', 'param_min_samples_leaf', 'param_min_samples_split', 'param_splitter', 'mean_test_score']]
fig, axs = plt.subplots(1, len(dt_hp), figsize=(20,5), sharey='all')
for i, hp in enumerate(dt_hp):
dt_hpt_selected.groupby(hp)['mean_test_score'].mean().plot.bar(ax=axs[i], ylim=(0, 1.0))
axs[i].tick_params(labelrotation=0)
fig.suptitle('Average mean_test_score for Hyperparameter Settings for Decision Tree')
plt.show()
Random Forests
rf_hp = ['param_bootstrap', 'param_max_features', 'param_min_samples_leaf', 'param_min_samples_split', 'param_n_estimators']
rf_hpt_selected = rf_hpt_df[['param_bootstrap', 'param_max_features', 'param_min_samples_leaf', 'param_min_samples_split', 'param_n_estimators', 'mean_test_score']]
fig, axs = plt.subplots(1, len(rf_hp), figsize=(20,5), sharey='all')
for i, hp in enumerate(rf_hp):
rf_hpt_selected.groupby(hp)['mean_test_score'].mean().plot.bar(ax=axs[i])
axs[i].tick_params(labelrotation=0)
fig.suptitle('Average mean_test_score for Hyperparameter Settings for Random Forest')
plt.show()
Final Models Comparison
models = pd.DataFrame({
'Model': ['Logistic Regression', 'Decision Tree', 'Random Forests'],
'Score': [lr_acc, dt_acc, rf_acc]})
models.sort_values(by = 'Score', ascending = False)
sns.set_style('whitegrid')
plt.figure(figsize=(16,8))
plt.ylabel("Accuracy %")
plt.xlabel("Algorithms")
sns.barplot(x=models['Model'],y=models['Score'])
plt.title("Comparison of Accuracy Scores for Models Used")
plt.show()