Optimization Project / Car price prediction

Optimization - Linear Regression from scratch pt.2

import numpy as np import pandas as pd

Import the dataset

https://www.kaggle.com/nehalbirla/vehicle-dataset-from-cardekho

df = pd.read_csv("/work/Car details v3.csv")

As we can see, we have 13 different columns containing some car caractéristics. There are Categorical variables, but also Numerical values.

df.shape

df.head()

Pre-processing operations

Missing values management

df.isnull().any()

deletion of the null values.

df = df.dropna().reset_index(drop=True)

df.isnull().any()

Removing units

Here, we remove the units in order to handle the column as float columns

def remove_unit(df,colum_name) : t = [] for i in df[colum_name]: number = str(i).split(' ')[0] t.append(float(number)) return t

df['engine'] = remove_unit(df,'engine') df['mileage'] = remove_unit(df,'mileage') df['max_power'] = remove_unit(df,'max_power')

df.head()

Then we remove unusable variables

df.drop(['name' ,'torque'], axis=1,inplace=True)

df.head()

Keeping only the main fuel categories

df['fuel'].value_counts()

df = df[df.fuel != 'CNG'] df = df[df.fuel != 'LPG']

Convert str types columns to int

ColumsToConvert = [column for column in df.columns if df[column].dtype == 'O'] ColumsToConvert

We use get dummies in order to keep only numerical values in our dataframe

drop_first = true allows us to get a better result, we can store the same informations but with 4 columns less

def transform_col(df,Col): for i in Col: c = pd.get_dummies(df[i],prefix= i, drop_first =True) df = pd.concat ([df,c], axis = 1) df.drop(i , axis =1, inplace = True) return df

df = transform_col(df,ColumsToConvert)

df.columns

df.head()

Split our dataframe in two. with:

x ==> data

y ==> label

x = df.drop('selling_price' , axis =1) y = df.iloc[:,1]

y = y.values x = x.values # y = y.to_numpy() # x = x.to_numpy()

Only the values in the DataFrame will be returned, the axes labels will be removed.

Data standardization

We standardize x, in order to eliminate order of magnitude differences between the values of the different columns

from sklearn.preprocessing import StandardScaler scaler = StandardScaler() X = scaler.fit_transform(x)

Train Test split

Our test dataframe represents 25%

from sklearn.model_selection import train_test_split Xtrain, Xtest, Ytrain, Ytest = train_test_split(X,y, test_size= 0.25, random_state=42)

Models creation

First, Let's try to implement some models in order to highlight the most efficient

from sklearn.metrics import mean_squared_error, mean_absolute_error from sklearn.model_selection import cross_val_score from sklearn.model_selection import validation_curve from sklearn.metrics import r2_score ModelList = []

1 - Linear Regression (67%)

from sklearn.linear_model import LinearRegression modelLR = LinearRegression() modelLR.fit(Xtrain,Ytrain) modelLR.score(Xtest,Ytest)

np.mean(cross_val_score(LinearRegression(), Xtrain, Ytrain, cv=10))

We can see that the model is not overfitting

https://fr.wikipedia.org/wiki/Validation_crois%C3%A9e

2 - DecisionTreeRegressor (94.6%)

from sklearn.tree import DecisionTreeRegressor modelDTR = DecisionTreeRegressor() modelDTR.fit( Xtrain, Ytrain)

modelDTR.score(Xtest,Ytest)

np.mean(cross_val_score(DecisionTreeRegressor(), Xtrain, Ytrain, cv=10))

3 - Ridge regression with optimized parameters (67%)

from sklearn.model_selection import GridSearchCV from sklearn.linear_model import Ridge modelRi = Ridge() parametersRidge = {'alpha': range(1,200)} RidgeGrid= GridSearchCV(modelRi, parametersRidge,cv=10)

RidgeGrid.fit(Xtrain,Ytrain)

RidgeGrid.score(Xtest,Ytest)

RidgeGrid.best_params_

np.mean(cross_val_score(Ridge(alpha= 191), Xtrain, Ytrain, cv=10))

4 - K Neighbors Regressor with optimized parameters (95%)

from sklearn.neighbors import KNeighborsRegressor parametersKNN = [{'weights': ['uniform', 'distance'], 'n_neighbors': range(1, 10, 1), 'algorithm': ['ball_tree', 'kd_tree', 'brute'], 'p': range(1, 3, 1)}] modelKNN =KNeighborsRegressor() KNNGrid= GridSearchCV(modelKNN, parametersKNN,cv=10)

modelKNN.fit(Xtrain,Ytrain)

np.mean(cross_val_score(KNeighborsRegressor(), Xtrain, Ytrain, cv=12))

KNNGrid.fit(Xtrain,Ytrain)

KNNGrid.best_params_

np.mean(cross_val_score(KNeighborsRegressor(n_neighbors=3,weights='distance',algorithm='kd_tree', p=1), Xtrain, Ytrain, cv=12))

5 - Stochastic Gradient Descent (67%)

from sklearn.linear_model import SGDRegressor modelSGD = SGDRegressor() modelSGD.fit(Xtrain,Ytrain) modelSGD.score(Xtest,Ytest)

np.mean(cross_val_score(SGDRegressor(), Xtrain, Ytrain, cv=10))

6 - Random Forest (96%)

from sklearn.ensemble import RandomForestRegressor modelRFR = RandomForestRegressor() modelRFR.fit(Xtrain,Ytrain) modelRFR.score(Xtest,Ytest)

np.mean(cross_val_score(RandomForestRegressor(), Xtrain, Ytrain, cv=10))

7 - Gradient Boosting (95 %)

from sklearn.ensemble import GradientBoostingRegressor modelGBR = GradientBoostingRegressor() modelGBR.fit(Xtrain, Ytrain) modelGBR.score(Xtest,Ytest)

np.mean(cross_val_score(GradientBoostingRegressor(), Xtrain, Ytrain, cv=10))

8 - XG Boost (96.5%)

!pip install xgboost==1.3.3

from xgboost import XGBRegressor modelXGB = XGBRegressor() modelXGB.fit(Xtrain, Ytrain) modelXGB.score(Xtest,Ytest)

np.mean(cross_val_score(XGBRegressor(), Xtrain, Ytrain, cv=10))

9 - Extra Tree Regressor (96.5%)

from sklearn.ensemble import ExtraTreesRegressor modelETR = ExtraTreesRegressor() modelETR.fit(Xtrain, Ytrain) modelETR.score(Xtest,Ytest)

np.mean(cross_val_score(ExtraTreesRegressor(), Xtrain, Ytrain, cv=10))

ETRGrid = GridSearchCV( cv=5, estimator=modelETR, param_grid={ 'n_estimators': range(50,200, 10), 'max_features': ['sqrt', 'log2', 'auto'], 'min_samples_leaf': range(1,3,1), 'min_samples_split': range(2,10,1), } ) ETRGrid.fit(Xtrain,Ytrain)

ETRGrid.best_params_

modelETR1 = ExtraTreesRegressor(min_samples_leaf = 1, min_samples_split = 3,n_estimators = 100) modelETR1.fit(Xtrain, Ytrain) modelETR1.score(Xtest,Ytest)

Optimization - Linear Regression from scratch pt.2

Import the dataset

Pre-processing operations

Missing values management

Removing units

Data standardization

Train Test split

Models creation

1 - Linear Regression (67%)

2 - DecisionTreeRegressor (94.6%)

3 - Ridge regression with optimized parameters (67%)

4 - K Neighbors Regressor with optimized parameters (95%)

5 - Stochastic Gradient Descent (67%)

6 - Random Forest (96%)

7 - Gradient Boosting (95 %)

8 - XG Boost (96.5%)

9 - Extra Tree Regressor (96.5%)

Recommended on Deepnote

Stock market analysis

10 ways to create NumPy arrays

Wide Residual Networks