Linear Regression using Gradient Descent

Linear Regression by implementing of gradient descent algorithm

The linear regression line is defined as $$y = \theta_0 + \theta_1 x $$

The parameters $\theta_0$ and $\theta_1$ can be computed using gradeint descent algorithm.

Gradeint descent algorithm is given as:

repeat until convergence { \begin{align} \theta1 &= \theta_1 - \alpha \frac{1}{m}\sum{i=1}^{m} \big ( h_\theta (x^{(i)}) - y^{(i)} \big ) x^{(i)} \ \theta0 &= \theta_0 - \alpha \frac{1}{m}\sum{i=1}^{m} \big ( h_\theta (x^{(i)}) - y^{(i)} \big ) \end{align*} }

Reference: https://towardsdatascience.com/

Step 1: Import libraries and dataset

## Import all the necessary libraries import numpy as np import matplotlib.pyplot as plt import pandas as pd %matplotlib inline

## Import the dataset data = pd.read_csv('salary_data.csv') # change the path as necessary X = data.iloc[:, 0] Y = data.iloc[:, 1] # View the size of the arrays X and Y print(X.shape) print(Y.shape)

# Visualize the dataset plt.scatter(X, Y, color='blue') plt.title('Salary VS Experience (Observations)') plt.xlabel('Year of Experience') plt.ylabel('Salary') plt.show()

Step 2 is skipped.

Step 3: Create and train the machine learning model

# Build the model theta1 = 0 theta0 = 0 alpha = 0.0001 # Learning Rate epochs = 10000 # Number of iterations to perform gradient descent m = float(len(X)) # Number of elements in X cost_history = [] # Performing Gradient Descent for i in range(epochs): Y_pred = theta1 * X + theta0 temp1 = (-1/m) * sum(X * (Y - Y_pred)) temp0 = (-1/m) * sum(Y - Y_pred) theta1 = theta1 - alpha * temp1 theta0 = theta0 - alpha * temp0 cost = (1/2*m) * sum((Y - Y_pred)**2) cost_history.append(cost)

Step 4: Visualize the results

# The coefficients # print the parameter theta1 print('Theta1 = ', theta1) # print the parameter theta0 print('Theta0 = ', theta0)

Step 5: Prediction

# Predict the values for the given X Y_pred = theta1 * X + theta0

# Visualize the dataset and plot the residuals fig, ax = plt.subplots() ax.scatter(X, Y, color='blue') # observed values ax.scatter(X, Y_pred, color='green') # predicted values ax.vlines(X,Y, Y_pred, color='red') # residual lines plt.plot([min(X), max(X)], [min(Y_pred), max(Y_pred)], color='black') # regression line plt.title('Salary VS Experience') plt.xlabel('Year of Experience') plt.ylabel('Salary') plt.show()

# plot the cost function plt.plot(cost_history) plt.title('Cost Function using Gradient Descent') plt.xlabel("Number of iterations") plt.ylabel("Cost") plt.show()

Step 6: Performance measures

from sklearn.metrics import mean_squared_error, r2_score # The mean squared error print("Mean squared error = %.2f" % mean_squared_error(Y, Y_pred)) # Explained variance score: 1 is perfect prediction print('Variance score = %.2f' % r2_score(Y, Y_pred))

Linear Regression using Gradient Descent

Linear Regression by implementing of gradient descent algorithm

Step 1: Import libraries and dataset

Step 3: Create and train the machine learning model

Step 4: Visualize the results

Step 5: Prediction

Step 6: Performance measures

Recommended on Deepnote

Stock Market Analysis

The 10 Best Ways to Create NumPy Arrays

Wide Residual Networks