Python Project 5 - K-Means Clustering

In this project we want to explore a dataset which contains information on breakfast cereals. We are only going to consider the manufacturers, cereal ratings, sugar and carbohydrate levels. Some of the values in sugars and carbohydrate features have invalid data which we must address.

We want to answer the following questions.

What cereal manufacturers have the most/least varieties?
What cereal has the highest rating?
Investigate sugar levels in cereals

Does each manufacturer provide cereals equally among the sugar levels?
How are the cereals clustered by sugar levels (low, medium, high)
What are the cereals which have the highest sugar level? lowest sugar levels?
How does your favorite breakfast cereal rank?

Investigate carbohydrate levels

Cluster cereals in low, medium and high level of carbs
What cereals are low sugar and low carbs?
What cereals are high sugar and high carbs?

Steps:

Input libraries and K-Means model from scikit-learn
Create dataframe and print out a few lines
Drop all features in dataframe except those we are using: Name, Manufacturer, Sugars and Carbohydrates, and Rating; alternately you can create a new dataframe with just the features you want
Check to see if there are any NaN values in sugars or carbohydrates features
Check to see if there are any invalid values (that is values <0) in sugars or carbohydrates features; if there are 2 or less cereals with these invalid values drop the cereal(s) from dataframe; otherwise replace negative values with 0.
Plot number of cereals for each manufacturer; if any manufacturer has only 1 cereal, drop that data instance. Which manufacturer has the most varieties of cereals.
In dataframe the manufacturer is listed by a single letter; for plotting purposes add a column giving the actual name and delete feature 'mfr'
Use a plot to determine which manufacturer has cereals with the highest ratings.
What cereal has the highest rating?
Plot sugars levels vs manufacturer and determine which brand has lowest sugar levels.
Cluster data using K-Means and the sugars feature with clusters low, middle and high sugar levels; print out cluster centroids; add cluster as feature in dataframe.
Determine which cluster is associated with low, mid or high sugar levels; add feature giving sugar level corresponding to cluster
Create a plot to show how cereals are distributed among sugar levels; what cereals have highest sugar levels? lowest sugar levels?
Repeat the steps you did for investigating sugar levels using carbohydrate levels.
Print out the cereals that are low sugars and low carbohydrates.
Print out the cereals that are low sugars and high carbohydrates.

Data Set Description

This dataset is contained in the file `cereal.csv'

This dataset contains information on 77 different breakfast cereals. The features are

Cereal Name
Manufacturer

A -> American Home Food Products
G -> General Mills
K -> Kelloggs
N -> Nabisco
P -> Post
Q -> Quaker Oats
R -> Ralston Purina

Type (C -> Cold, H -> Hot )
Calories (per serving)
Protein (in grams)
Fat (in grams)
Sodium (in milligrams)
Fiber (in grams of dietary fiber)
Carbo (grams of complex carbohydrates)
Sugars (in grams)
Potass (milligrams of potassium)
Vitamins (possible values are 0, 25 or 100 indicating percent of FDA recommended vitamins and minerals)
Shelf ( display shelf with possible values 1, 2, or 3 counting from floor)
Weight (weight of one serving in ounces)
Cups (number of cups in one serving)
Rating (Consumer ratings of cereal from 0 to 100)

# import libraries and K-Means function import numpy as np import matplotlib.pyplot as plt from pandas import DataFrame import pandas as pd import seaborn as sns from sklearn.neural_network import MLPClassifier from sklearn.neighbors import KNeighborsClassifier from sklearn.linear_model import LogisticRegression from sklearn.cluster import KMeans

# Create dataframe for data df = pd.read_csv ('cereal.csv') df

# drop features we aren't using or create a new dataframe with only the features we want #Name, Manufacturer, Sugars and Carbohydrates, and Rating df=df.drop(columns=['type', 'calories','protein','fat','sodium','fiber','potass','vitamins','shelf','weight','cups']) df.head()

# Check to see if any instances have NaN for entries using .info df.info()

# Check to see if any cereals have negative values for sugars or carbohydrates; if 2 or less cereals, drop those # data instances; otherwise replace negative values with 0 df[df.sugars < 0] df[df.carbo <0]

# Address negative values #we dropped Quaker Oatmeal, row 57 which had negative sugars and carbs df = df.drop([57]) print(type (df.sugars)) df.sugars

# set background grid for plots sns.set_style( 'whitegrid')

# Plot number of products per manufacturer #introduction seaborn part 1, use account plot x axis manufacturer sns.countplot (x='mfr', data = df) #Manufacturer K Kelloggs has the most amount of cereals

# There is only one cereal from American Home Foods Company so we drop that data sample # # Type df[df.mfr == 'A'] and it will give you all data from this manufacturer. # Make sure you set df = drop command; use command to make sure it was deleted # df[df.mfr == 'A'] df = df.drop([43]) sns.countplot (x='mfr', data = df)

# For plots we would like the name of manufacturer instead of just "N" or "Q" # Use .map or .apply # # Then drop 'mfr' column # def name (mfr) : if (mfr =='N'): return('Nabisco') elif (mfr =='Q'): return('Quaker Oats') elif (mfr =='K'): return('Kellogs') elif (mfr=='R'): return('Ralston Purina') elif (mfr =='G'): return('General Mills') elif (mfr=='P'): return('Post') else: return("NA") df['Manufacturer']=df['mfr'].apply (name) df = df.drop(columns=['mfr']) df.head()

# Plot to see which manufacturer has highest rated cereals sns.catplot(x='Manufacturer', y='rating',data = df ) plt.xticks(rotation=-45)

# Which cereal is rated highest? column = df['rating'] max_value = column.max() print(max_value) df [df.rating == 93.704912] #all-Bran with Extra Fiber is the cereal with the highest rating

# Look at sugars per brand by plotting sns.catplot(x='Manufacturer', y='sugars',data = df ) plt.xticks(rotation=-45)

Cluster by sugars into highest, middle and lowest levels; random initial guess

# Get data for clustering X = df.sugars.values n = len(X) X = np.reshape (X, (n,1)) km = KMeans( n_clusters=3, init='random', random_state=0 ) y_km = km.fit(X)

# Form model, fit data and print out cluster centers print ("Cluster centers for sugar clusters") print() print (km.cluster_centers_ )

# Add column to dataframe for this clusters, say sugars_clusters df['sugar_clusters'] = km.labels_ df.head()

# Plot clusters sns.catplot (x= 'sugar_clusters', y='sugars', data = df) plt.xticks(rotation=45)

# Determine which cluster number corresponds to lowest, middle and highest level and create a new # column in dataframe using .map # df['sugar_clusters_levels'] = df['sugar_clusters'].map (\ {0: 'low sugar', 1: 'mid sugar',\ 2: 'high sugar'}) # df.head()

# How are cereals distributed among the 3 levels? sns.catplot(x='Manufacturer', y='sugar_clusters_levels',data = df ) plt.xticks(rotation=-100)

# Which cereals have the highest sugar levels # df[(df.sugar_clusters ==2)]

# Which cereals have the lowest sugar levels # df[(df.sugar_clusters ==0)]

# If you eat a particular cereal like Apple Jacks, Froot Loops, etc. what cluster is it in? my_cereal = 'Apple Jacks' print (f"The data instance and sugar cluster for {my_cereal} is ", \ df[df.name == my_cereal].sugar_clusters )

Repeat calculations and plots for carbohydrates instead of sugars

# Get data for clustering carbs X = df.carbo.values n = len(X) X = np.reshape (X, (n,1)) km = KMeans( n_clusters=3, init='random', random_state=0 ) y_km = km.fit(X) # Form model, fit data and print out cluster centers print ("Cluster centers for carb clusters") print() print (km.cluster_centers_ )

# Add column to dataframe for this clusters, say carbo_clusters df['carbo_clusters'] = km.labels_ df.head()

# Plot clusters sns.catplot (x= 'carbo_clusters', y='carbo', data = df) plt.xticks(rotation=45)

# Determine which cluster number corresponds to lowest, middle and highest level and create a new # column in dataframe using .map # df['carbo_clusters_levels'] = df['carbo_clusters'].map (\ {0: 'mid carbs', 1: 'low carbs',\ 2: 'high carbs'}) # df.head()

# How are cereals distributed among the 3 levels? sns.catplot(x='Manufacturer', y='carbo_clusters_levels',data = df ) plt.xticks(rotation=-100)

# Which cereals have the highest carb levels # df[(df.carbo_clusters ==2)]

# Which cereals have the lowest carb levels # df[(df.carbo_clusters ==1)]

What cereals are high carbs and low sugar? What are low carbs and low sugar?

#cereals that are high carbs and low sugar df[ (df.carbo_clusters ==2) & (df.sugar_clusters==0)]

#Cereal that are low carbs and low sugar df[ (df.carbo_clusters ==1) & (df.sugar_clusters==0)]

Python Project 5 - K-Means Clustering

Data Set Description

Repeat calculations and plots for carbohydrates instead of sugars

Recommended on Deepnote

Stock Market Analysis

The 10 Best Ways to Create NumPy Arrays

Wide Residual Networks