Module 1 Homework

Charles Gagnon, U00316286

Download Anaconda for Python or R
Copy the data in Problem 2.8 into a CSV file
Import your CSV file and create a dataframe
Describe your dataframe (statistically and in words)
Normalize your dataframe
Create a scatter plot of your data. Include both the interpolated and regular plots.
Split your data into 60% training and 40% validation
Upload your Jupyter notebook or simply a text file of your code.

# Imports import pandas as pd

Next we import "prob_2.8.csv" in the /income/ dataframe. The data in the CSV is from Table 2.18 (Shmueli).

Field	Description
AGE	Age in years
INCOME	Income in US dollars (USD)

References:

Shmueli, Galit,Bruce, Peter C.,Gedeck, Peter,Patel, Nitin R.. Data Mining for Business Analytics (Kindle Locations 1595-1597). Wiley. Kindle Edition.

# income dataframe income_df = pd.read_csv('prob_2.8.csv') income_df.describe()

# We'll need the DataFrame sorted by age income_df = income_df.sort_values('Age') income_df

# Normalizing a data frame income_df['Norm_Income'] = (income_df['Income'] - income_df['Income'].mean()) / income_df['Income'].std() income_df

## scatter plot with axes names income_df.plot.scatter(x='Age', y='Income', legend=False)

x = list(income_df.Age) y = list(income_df.Income) import numpy as np from scipy import interpolate import matplotlib.pylab as plt f = interpolate.PchipInterpolator(x, y) x_new = np.linspace(x[0], x[-1], 100) y_new = [f(xi) for xi in x_new] income_df.plot.scatter(x='Age', y='Income') plt.plot(x_new, y_new) plt.tight_layout() # Increase the separation between the plots plt.show()

from sklearn.model_selection import train_test_split # training (60%) and validation (40%) trainIncome, validIncome = train_test_split(income_df, test_size=0.40, random_state=1) print('Training : ', trainIncome.shape) print('Validation : ', validIncome.shape)

Module 1 Homework

Recommended on Deepnote

Stock Market Analysis

The 10 Best Ways to Create NumPy Arrays

Wide Residual Networks