A/B testing

This notebook visualizes and measures the results of an A/B test.

Measure weekly conversions from visiting the landing page to signing up.

Measure weekly retention of users who signed up.

Conversion rate

We need to check whether one variant resulted in a higher conversion rate than another. Let's start with a time series.

Here is the overall conversion rate for each variant:

Significance test

It looks like Variant B has a higher conversion rate than Variant A. We need to make sure these results are significant.

Retention rate

We also want to know whether one variant of the website results in higher retention than another variant. For this, let's look at the weekly sessions of our users.

Significance test

Once again, check that the difference in retention is significance.