MapReduce notebook: max transaction value by store

This notebook focuses on MapReduce, which processes large datasets. It is designed to find the maximum transaction value by store using sales data. For those who want to learn MapReduce from scratch, this notebook covers the basics. For more information, here is a detailed article.

1. Data exploration

In this section, the sales data are explored to gain insights into transaction values.

# Summary statistics df.describe() # Group by Store and calculate statistics store_stats = df.groupby('Store')['Cost'].describe() store_stats

2. Finding maximum transaction per store

This code snippet shows a simplified MapReduce process to find the maximum transaction value for each store. The mapper function extracts the Store and Cost columns, while the reducer groups the data by Store and calculates the maximum Cost for each store. The results display the highest transaction values per store, showing the efficiency of MapReduce in data analysis.

# Function to simulate the Mapper step def mapper(dataframe): return dataframe[['Store', 'Cost']] # Function to simulate the Reducer step def reducer(mapped_df): return mapped_df.groupby('Store')['Cost'].max().reset_index() # Apply MapReduce mapped_df = mapper(df) reduced_df = reducer(mapped_df) # Display results reduced_df.head()

3. Visualizations

3.1 Maximum transaction per store & total share of transaction cost by store

In the bar chart left below, the maximum transaction cost for each store is displayed. The second histogram shows how to the transaction costs are distributed.

3.2. Distribution of transaction costs