Basics of Machine Learning

Julián Cárdenas

In this example we create a machine learning model that predicts the value of houses from some parameters ('SqFt', 'Bedrooms', 'Bathrooms', 'Offers)

Basic Data Exploration

import pandas as pd

df_path = 'https://raw.githubusercontent.com/ywchiu/riii/master/data/house-prices.csv' df = pd.read_csv(df_path) df.head()

Homeint64

Priceint64

114300

114200

114800

94700

119800

df.shape

df.groupby('Neighborhood').count()

Homeint64

Priceint64

East

North

West

#Dont have null df.isnull().count()

df.describe()

Homefloat64

Pricefloat64

count

128.0

mean

64.5

130427.34375

std

37.094473981982816

26868.770370734055

min

1.0

69100.0

25%

32.75

111325.0

50%

64.5

125950.0

75%

96.25

148250.0

max

128.0

211200.0

First Machine Learning Model

#In case of missing values df = df.dropna(axis=0)

#In this case i want to predict the price y = df.Price

#In this case im going to work with the relevant numeric features of the dataframe df.columns

df_features = ['SqFt', 'Bedrooms', 'Bathrooms', 'Offers']

X = df[df_features] X.describe()

SqFtfloat64

Bedroomsfloat64

count

128.0

mean

2000.9375

3.0234375

std

211.5724313344898

0.7259513850016589

min

1450.0

2.0

25%

1880.0

3.0

50%

2000.0

3.0

75%

2140.0

3.0

max

2590.0

5.0

X.head()

SqFtint64

Bedroomsint64

1790

2030

1740

1980

2130

#1. Define the type of model #https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html from sklearn.tree import DecisionTreeRegressor df_model = DecisionTreeRegressor(random_state=1)

#2. Fit df_model.fit(X, y)

#3. Predict print("Making predictions for the following 5 houses:") print(X.head()) print("The predictions are") print(df_model.predict(X.head()))

Making predictions for the following 5 houses:
   SqFt  Bedrooms  Bathrooms  Offers
0  1790         2          2       2
1  2030         4          2       3
2  1740         3          2       1
3  1980         3          2       3
4  2130         3          3       3
The predictions are
[114300. 114200. 114800.  94700. 119800.]

#4. Evaluate R^2 #https://scikit-learn.org/stable/modules/generated/sklearn.metrics.r2_score.html #R^2: Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). from sklearn.metrics import r2_score #r2_score(true_values, predict_values) r2_score(y, df_model.predict(X))

#4. Evaluate MAE #So, if a house cost $150,000 and you predicted it would cost $100,000 the error is $50,000. #https://scikit-learn.org/stable/modules/generated/sklearn.metrics.mean_absolute_error.html from sklearn.metrics import mean_absolute_error #mean_absolute_error(true_values, predict_values) mean_absolute_error(y, df_model.predict(X))

Train test split

The Problem with "In-Sample" Scores The measure we just computed can be called an "in-sample" score. We used a single "sample" of houses for both building the model and evaluating it. Here's why this is bad.

Imagine that, in the large real estate market, door color is unrelated to home price.

However, in the sample of data you used to build the model, all homes with green doors were very expensive. The model's job is to find patterns that predict home prices, so it will see this pattern, and it will always predict high prices for homes with green doors.

Since this pattern was derived from the training data, the model will appear accurate in the training data.

But if this pattern doesn't hold when the model sees new data, the model would be very inaccurate when used in practice.

Explanation from: https://www.kaggle.com/code/dansbecker/model-validation

#https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html from sklearn.model_selection import train_test_split train_X, val_X, train_y, val_y = train_test_split(X, y, random_state = 0)

# Define model df_model = DecisionTreeRegressor() #Fit model df_model.fit(train_X, train_y) #Get predictions prices on validation data val_predictions = df_model.predict(val_X) print(mean_absolute_error(val_y, val_predictions))

20118.75