Customer churn prediction

Image Source: What Is Churn Rate and Why It Will Mess With Your Growth

1. Introduction

Customer Churn is when customers leave a service in a given period of time, what is bad for business.
This work has as objective to build a machine learning model to predict what customers will leave the service, the dataset used on this notebook is the Telco Customer Churn hosted at Kaggle. Also, an Exploratory Data Analysis is made to a better understand about the data. Another point on this work is use Deepnote as development enviroment and the PyCaret Python Module to make all the experiment pipeline.

1.1 Enviroment Setup

The Modules used for this work, highlights for PyCaret and good plots by Plotly.

And the helper functions used on this notebook.

2. Load Data

The Dataset is load as a Pandas dataframe and show a gimplse of the data. A good thing about Deepnote is that the displayed dataframes shows the column type, helping to understand the features.

Check for duplicated samples.

Is needed so more information about the dataset as the number of samples, memory size allocation, etc.
The result is showed on the following output (The Data Type is just showed for convenience, to make this notebook useful on other enviroments).

The dataset has a small memory size allocation (1.183 MB) and is composed for many Categorical (object) features and only a few numeric, but one of the categorical features doesn't look right, the TotalCharges, as showed on the displayed dataframe, the festure is numeric.
TotalCharges is converted from Object to float64, the same of MonthlyCharges feature.

3. Exploratory Data Analysis

3.1 Churn Distribution

The Client Churn Distribution is checked for any imbalance, as the feature is the target, it's important to choose what strategy to adopt when dealing with imbalanced classes.
Below, a Pie Chart shows the feature distribution.

There's some imbalance on Churn Distribution, 26.5% of the clients have churned, and small occurences of a label could lead to bad predictor.
It's possible to choose some ways to work with this case:

• Make a random over-sampling, duplicating some samples of the minority class until this reach a balance, but this could lead to an overfitted model.
• Make a random down-sampling, removing some samples from the majority class until this reach a balance, but this leads to information loss and not feeding the model with the collected samples.
• Make a random down-sampling, removing some samples from the majority class until this reach a balance, but this leads to information loss and not feeding the model with the collected samples.
• Another resampling technique, as SMOTE.
• Choosing a metric that deals with imbalanced datasets, like F1 Score.

The Churn problem is about client retention, so is worth to check about false positives, so precision and recall metrics are a must for this situtation.
F1 Score is used to check the quality of the model predictions, as the metric is an harmonic mean of precision and recall.

3.2 Analysis of the Contract Type

The contract type is a good feature to analyze what happens to a client churn from that service, a plot from the contract types of not churned clients is showed below.

Is showed that a Month-to-month contract is the firts when compared to annual contracts, but the difference between the number of contracts is not so big.
To a better comparation, the same plot is showed for the churned clients.

Now, the difference between a Month-to-month and annual contractts is bigger, and can lead to a conclusion that annual contracts are better to retain the clients, perhaps fidelity promotions could aid to reduce the churn rate.
As the problem can be examined more deep on Month-to-month contract types, a good idea is see the Monthly Charges and Total Charges distribution for the not churned clients of this contract.

From the plots, can be said that many clients just got charged with a few values, principally for the Total Charges.
On the following plots, the same features are analyzed, but for churned clients.

Total Charges had the same behaviour, but the Monthly Charges for many churned clients was high, maybe the amount of chage value could lead the client to leave the service.
Still on the Month-to-month contract, it's time to analyze the most used Payment methods of churned clients.

Many Churned Clients used to pay with electronic checks, automatic payments, as bank transfers or credit card have a few churned clients. A good idea could make promotions to clients that use automatic payment methods.
Lastly, the tenure of the churned clients.

Most clients just used the service for one month, seems like the clients used to service to check the quality or the couldn't stay for the amount of charges, as the Monthly Charges for these clients was high and the Total Charges was small, as the client just stayed a little time.

4. Setting up PyCaret

Before setting up PyCaret, a random sample of 10% size of the dataset will be get to make predictions with unseen data.

The PyCaret's setup is made with 90% of data samples and just use one function (setup) from the module.
It's possible configure with variuos options, as data pre-processing, feature engineering, etc. The easy and efficient of PyCaret buy a lot of time when prototyping models.
Each setup is an experiment and for this problem, is used the following options:

• Normalization of the numerical features with Z-Score.
• Feature Selection with permutation importance techniques.
• Outliers Removal.
• Features Removal based on Multicollinearity.
• Features Scalling Transformation.
• Ignore low variance on Features.
• PCA for Dimensionality Reduction, as the dataset has many features.
• Numeric binning on the features MonthlyCharges and TotalCharges.
• 70% of samples for Train and 30% for test.

PyCaret shows at first if all features types are with it correspondent type, if everything is right, press enter on the blank bar and the setup is finished showing a summary of the experiment.

5. Model Build

A great tool on PyCaret is build many models and compare a metric for the bests!
Due the target class imbalance, the models are sorted by F1 Score.

The best model suggested by PyCaret is the Quadratic Discriminant Analysis (QDA), with a F1 Score around 0.6 and a good Recall, around 0.7.
Let's stick with QDA and create the model.

And see the hyper-parameters used for build the base model.

It's possible to tune the base model and optmize a metric, for this case, F1 Score.

There's an improvement from the base model on F1 Score! Now, time to see what hyper-parameters were used by the tuned model.

The reg_param was higher on the tuned model.
The Only problem with using QDA is that not possible to get the Features Importance plot of the model, but the model is used as some good insights about the data were got on the EDA.
PyCaret also has functions to make ensembles, for this implementation, a bagged model is build.

The bagged model improved a bit the F1 Score, it's also possible make blended and stacked models with PyCaret, both models are created using the the tuned and bagged models with the soft method.

Although the stacked model had improved precision on a good amount, the blended still got a better F1 and it is saved as the best model.
This model is saved and its ROC Curves are plotted below.

6. Prediction on Test Data

The test is made with the remaining 30% of data that PyCaret got on the setup, it's important to see that the model is not overfitting.

As everything is right with the model, it's time to finalize it fitting all the data.

7. Prediction on Unseen Data

The remaining 10% data is used to make predictions with unseen samples, what could include some outliers, it's how real world data works.
Just Kappa Score is not showed, as the focus is the F1 Score.
It's not necessary to make any transformation on the data, PyCaret do this.

And the Unseen Data predicts as the trained model! The model was sucessful built!

8. Save Experiment and Model

PyCaret allows to save all the pipeline experiment and the model to deploy.
It's recommended to save with date of the experiments.

9. Conclusion

From the results and explanations presented here, some conclusion can be draw:

• The type of contract has a strict relationship with churned clients, Month-to-month contracts with high amount of charges could lead a client to leave the service.
• For the predictions made by the model and based on the precision and recall scores, as F1 Score try to show a balance between these two metrics, the precision was near 54%, what means that the model predict correctly 54% of classified clients as churned, on other hand, the recall was good, where around 80% of the actually churned clients was predict correctly.
• The metrics must be used in favor of the business interests, is needed a more correct prediction of the churned clients or get more a part of these clients?
• One possible way to improve the results is SMOTE as data resampling.

From the tools and enviroment used:

• Deepnote is a fantastic tool to use Python Notebooks, the design is clear, a good variable explorer, an option to publish the notebook as an article, terminal for commands like git, etc.
• PyCaret is incredible, it speed up the model build a lot and the pre-processing is very useful, beyond of the functions to save and deploy the model.

Thanks for you reading!