Homework 2: Data Cleaning Basics
In this homework, we will be going over a couple of concepts we learned in lecture 2! Everything in this homework was covered in lecture so feel free to reference the slides and remember you can also google!
import pandas as pd
import numpy as np
from datascience import *
movies = pd.read_csv("movies_dataset.csv")
movies.head(10)
Question 1:
#In the table above, there are nan values. Normally we would investigate more before
#removing these rows but for the sake of this homework let us just remove them! Write a line of code below
#in "nans_removed" to remove all nan values.
nans_removed = movies.dropna()
nans_removed
Question 2:
#Now that we've removed the nan values let's remove some columns. Suppose the columns: {"writer", "storyline",
# "industry"} don't interest us. Write a line of code in "columns_dropped" that removes these columns
columns_dropped = nans_removed.drop(columns=['writer', 'storyline', 'industry'])
columns_dropped
Question 3:
#We've decided to focus on more popular movies. So we want to only keep movies that have an
#IMDb-rating of at least 5. Write a line of code in "popular_only" to do this!
#Reference the lecture slides if you're stuck!
popular_only = columns_dropped[columns_dropped['IMDb-rating'] >= 5]
popular_only
Question 4
#For this question please respond with a text cell outlining the steps involved in Data Cleaning. #Please also include steps for what you would do with outliers, missing/NA values and wrong type values.
- For outliners, investigate then might remove them. - For missing / NA values or wrong values, I could try valrious methods to replace the the missing / NA by mean, median, or linear regression.
CONGRATS! You've finished the coding part of your homework. For the last part of your homework include a summary of 5 things you learned from this week's DSS's Article:
ChatGPT could be a great study buddy or essay reviewer.
Banning AI is not a solution because people always try to game the system.
Using AI while in school will prepare students for the real world.
Teachers can use ChatGPT to improve their materials.
There are other AIs that do the same as ChatGPT, and will be more of them. Banning is not the solution.