Data Analytics with Pandas 🐼

The Pandas package is the most imperative tool in Data Science and Analysis working in Python nowadays.

In this notebook, we would analyze Supermarket Data Across the Country; Company XYZ.

Alright, let’s start!

Step 1 - Loading the Dataset

import os import glob import pandas as pd import numpy as np path ="/work" os.chdir(path) #Current working directory that contains your dataset and code file

#Use glob to match the pattern "csv" extension = 'csv' result = glob.glob('*.{}'.format(extension)) print (result)

['Lagos_Branch.csv', 'Port_Harcourt_Branch.csv', 'Abuja_Branch.csv']

dir = "./" csv_files = [] for file in glob.iglob(dir + "/*."+extension, recursive=True):csv_files.append(file)

#Combine all the files in the generated list above and export to a CSV df = [] for file in csv_files: dff = pd.read_csv(file) df.append(dff) branch = pd.concat(df, axis = 0) branch.to_csv("Branch.csv") df = pd.read_csv('Branch.csv')

Step 2 - Data Exploration

It is an important pillar of data science, a critical step required to complete every project regardless of the domain or the type of data you are working with.

The dataset was explored using some built-in pandas function like; Numpy for carrying out numerical computations, pandas for making a dataframe object seaborn and matplotlib for visualizations. The .head() to view the first few observations of the dataframe and .info() for information about the dataframe.

# Import Libaries import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt %matplotlib inline plt.style.use('fivethirtyeight') import warnings warnings.filterwarnings('ignore')

df.head(3)

#Check the number of rows and columns present in the data using the shape attribute. df.shape

#are there missing values df.isna().sum()

#the names of the columns using the columns attribute. df.columns

df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1000 entries, 0 to 999
Data columns (total 18 columns):
 #   Column                   Non-Null Count  Dtype  
---  ------                   --------------  -----  
 0   Unnamed: 0               1000 non-null   int64  
 1   Invoice ID               1000 non-null   object 
 2   Branch                   1000 non-null   object 
 3   City                     1000 non-null   object 
 4   Customer type            1000 non-null   object 
 5   Gender                   1000 non-null   object 
 6   Product line             1000 non-null   object 
 7   Unit price               1000 non-null   float64
 8   Quantity                 1000 non-null   int64  
 9   Tax 5%                   1000 non-null   float64
 10  Total                    1000 non-null   float64
 11  Date                     1000 non-null   object 
 12  Time                     1000 non-null   object 
 13  Payment                  1000 non-null   object 
 14  cogs                     1000 non-null   float64
 15  gross margin percentage  1000 non-null   float64
 16  gross income             1000 non-null   float64
 17  Rating                   1000 non-null   float64
dtypes: float64(7), int64(2), object(9)
memory usage: 140.8+ KB

#Statiscal Summary df.describe()

Step 3 - Dealing with DateTime Features

import datetime as dt #Importing our datetime module df["Date"] = pd.to_datetime(df["Date"]) # Extracting the Dates df.dtypes

df["Time"] = pd.to_datetime(df["Time"]) #Extracting Time

df["Day"] = df["Date"].dt.day df["Month"] = df["Date"].dt.month df["Hour"] = df["Time"].dt.hour df["Year"] = df["Date"].dt.year

df["Hour"].nunique()

df["Hour"].unique()

df.columns

Step 4 - Unique Values in Columns

categorical_columns = [col for col in df.columns if df[col].dtype == "object" ] categorical_columns

df['Branch'].unique().tolist()

print("Total Number of unique values in the Branch Column : {}". format(len(df['Branch'].unique().tolist())))

Total Number of unique values in the Branch Column : 3

for i in categorical_columns: if i != 'Invoice ID': print ("This is for {} Coulmns".format(i)) print (df[i].unique().tolist())

This is for Branch Coulmns
['A', 'C', 'B']
This is for City Coulmns
['Lagos', 'Port Harcourt', 'Abuja']
This is for Customer type Coulmns
['Member', 'Normal']
This is for Gender Coulmns
['Female', 'Male']
This is for Product line Coulmns
['Health and beauty', 'Home and lifestyle', 'Sports and travel', 'Electronic accessories', 'Food and beverages', 'Fashion accessories']
This is for Payment Coulmns
['Epay', 'Card', 'Cash']

for i in categorical_columns: if i != 'Invoice ID': print ("This is for {} Coulmns".format(i)) print ("Total Number of unique value: {}". format(len(df[i].unique().tolist())))

This is for Branch Coulmns
Total Number of unique value: 3
This is for City Coulmns
Total Number of unique value: 3
This is for Customer type Coulmns
Total Number of unique value: 2
This is for Gender Coulmns
Total Number of unique value: 2
This is for Product line Coulmns
Total Number of unique value: 6
This is for Payment Coulmns
Total Number of unique value: 3

Step 5 - Aggregation with GroupBy

# Creating a groupby object to group the dataset according to City by_city = df.groupby("City") #To get the sum and mean of the numeric columns by_city.agg(["sum", "mean"])

# Using the groupby object ot get the gross income of each city gross_city = by_city['gross income'].sum().reset_index() gross_city

#Getting the city with the highest gross income gross_city.max().reset_index()

City_with_others = by_city["Unit price", "Quantity", "gross margin percentage"].sum() City_with_others

#Product line with the highest gross income prod = df[['Product line', 'gross income']] prod_group = prod.groupby('Product line').sum().reset_index().sort_values(['gross income'], ascending =False) prod_group

sales_quantity = df[['Branch','Quantity']] group = sales_quantity.groupby('Branch').count().sort_values(['Quantity'], ascending =False) group

Step 6 - Data Visualization

sns.countplot(x= 'Branch', data = df).set_title("Branch vs Sales", size = 24);

sns.countplot(x = 'Payment', data = df).set_title('Different payment card', size = 23);

sns.countplot(x = 'City', data = df).set_title('City Vs Sales');

sns.countplot(y = 'Product line', data = df).set_title('Sales in different product line');

sns.countplot(y = 'Product line', data = df, hue = 'Payment').set_title('Payment mode VS Product line');

sns.countplot(x = 'Payment', data = df, hue = 'Branch').set_title('Payment mode VS Branch') plt.legend(loc='upper left', bbox_to_anchor=(1.25, 0.5), ncol=1)

sns.boxplot(x = 'Branch', y = 'Rating', data = df).set_title("Rating vs Branch");

sns.catplot(x = 'Product line', y = 'Quantity', hue = 'Gender', kind= 'boxen', data = df, aspect = 4);

sns.catplot(y = 'Unit price', x = 'Product line', kind= 'point', data = df, aspect = 4);

sns.catplot(y = 'Quantity', x = 'Product line', kind= 'point', data = df, aspect = 4);

sns.countplot(x = 'Payment', data = df, hue = 'Gender').set_title("Gender vs Payment method");

sns.catplot(y = 'Tax 5%', x = 'City', kind= 'swarm', data = df, aspect =4);

sns.countplot(x = 'City', hue = 'Gender', data = df).set_title("Gender VS City");

Summary

In this tutorial, we have learnt how to perform Exploratory Data Analysis (EDA) in Python and how to use external Python packages such as Pandas, Numpy, Matplotlib, Seaborn etc. to conduct univariate analysis, bivariate analysis and data visualization.

from PIL import Image from PIL import GifImagePlugin imageObject = Image.open("tenorr.gif") size = (500,300) imageObject = imageObject.resize(size) imageObject.show()