GlutenGetOut

Automated Scanning: The application automatically scans uploaded webp files of food labels. These files are pre-uploaded and ready for analysis, ensuring a smooth and fast user experience.

Model Details: The machine learning model used for this application has been trained on a diverse dataset to detect gluten content with high accuracy. It analyzes the ingredients listed on the food labels and provides a prediction along with the confidence level of the prediction.

Review Results: Please verify the terminal outputs, one which contains the results of three (3) scanned images. Each result includes the input, the model’s prediction, and the confidence level as a measurement metric.

Download Files: The webp files used for scanning and the trained model and vectorizer are included in the provided zip file for your convenience.

Running the entire notebook will take approximately 4 minutes.

Due to the large size of the original dataset, branded_food.csv (912MB), this code is executable, taking approximately 40 seconds in the current notebook environment, and will create gluten_free_data.csv around 28MB.

import pandas as pd # Load Dataset file_path = '/datasets/brandedfood/branded_food.csv' full_df = pd.read_csv(file_path) # Define Gluten-Containing Ingredients gluten_ingredients_list = [ 'barley', 'brewer\'s yeast', 'durum', 'einkorn wheat', 'emmer', 'farina', 'farro', 'graham', 'kamut', 'khorasan wheat', 'malt', 'rye', 'semolina', 'spelt', 'triticale', 'wheat', 'wheat berries', 'wheat starch', 'malt extract', 'malt flavor', 'malt syrup', 'wheat flour' ] # Label Gluten Ingredients def label_gluten_extended(ingredients): if isinstance(ingredients, str): for ingredient in gluten_ingredients_list: if ingredient in ingredients.lower(): return 1 return 0 # Handle Any Missing Data full_df['ingredients'] = full_df['ingredients'].fillna('') # Apply Label to the Column full_df['contains_gluten'] = full_df['ingredients'].apply(label_gluten_extended) # Sample 100,000 Ingredient Lists df_sampled = full_df.sample(n=100000, random_state=42) # Save 'ingredients' and 'contains_gluten' Columns to a New File sample_file_path = '/work/gluten_free_data.csv' df_sampled[['ingredients', 'contains_gluten']].to_csv(sample_file_path, index=False)

Run to view results

import numpy as np import pandas as pd from sklearn.feature_extraction.text import CountVectorizer from sklearn.linear_model import LogisticRegression from sklearn.model_selection import train_test_split from joblib import dump # Read the Data df = pd.read_csv('gluten_free_data.csv') # Replace NaN Values with an Empty String df['ingredients'] = df['ingredients'].replace(np.nan, '', regex=True) # Vectorize the Ingredients Text vectorizer = CountVectorizer() vectorized_features = vectorizer.fit_transform(df['ingredients']) # Split the Data into Training and Testing Sets x_train, x_test, y_train, y_test = train_test_split(vectorized_features, df['contains_gluten'], test_size=0.2, random_state=42) # Train Logistic Regression Model model = LogisticRegression(max_iter=200) model.fit(x_train, y_train) # Create Models Folder if it Doesn't Exist if not os.path.exists('Models'): os.makedirs('Models') # Save the Model dump(model, 'Models/model.joblib') # Save the Vectorizer dump(vectorizer, 'Models/vectorizer.joblib') # Evaluate the Accuracy of the Model/ Data Product print(f"\nModel accuracy on test set: {model.score(x_test, y_test)}")

Run to view results

Due to the nature of EasyOCR, building and deploying the OCR model can take some time. The notebook is designed to automatically process and display results for the sample images of ingredients in the terminal block under the code. The images have been optimized to webp format. The entire code block runtime is approximately 30 seconds

import easyocr from joblib import load import textwrap import os # Load the Model and Vectorizer model = load('Models/model.joblib') vectorizer = load('Models/vectorizer.joblib') # Create an Instance of EasyOCR Reader reader = easyocr.Reader(['en']) # Sample Images Directory sample_images_dir = '/work/Sample Images/' # List images in the sample_images_dir sample_images = [img for img in os.listdir(sample_images_dir) if img.endswith(('.webp'))] # Function to process the selected image def process_image(file_path): # Print Message to Indicate Processing print(f"\nProcessing the image: {file_path}. Please wait...") # Use EasyOCR to Extract Text ocr_result = reader.readtext(file_path) text_from_image = ' '.join([item[1] for item in ocr_result]) # Vectorize the Input vectorized_input = vectorizer.transform([text_from_image]) # Use the Model to Predict prediction = model.predict(vectorized_input) prediction_proba = model.predict_proba(vectorized_input) # Print Scanned Text From Image wrapped_text = textwrap.fill(text_from_image, width=80) print(f"\nInput:\n{wrapped_text}") # Print Prediction if prediction[0] == 1: confidence = prediction_proba[0][1] message = "contains gluten" else: confidence = prediction_proba[0][0] message = "does not contain gluten" print(f"\nThe ingredient list likely {message} with {confidence:.2%} confidence.") # Automatically process each image in the sample_images list for image in sample_images: image_path = os.path.join(sample_images_dir, image) process_image(image_path)

Run to view results

🔝 Please verify the terminal output above was reviewed, as it contains the results of three (3) scanned images, followed by the model stating the input and prediction. Lastly, confidence is displayed as a measurement metric.

import matplotlib.pyplot as plt # Bar Chart of Gluten vs. Gluten-Free Ingredient Lists df['contains_gluten'].value_counts().plot(kind='bar') plt.xlabel('Contains Gluten') plt.ylabel('Count per 100,000 Samples') plt.title('Gluten vs. Gluten-Free Ingredient Lists') plt.xticks([0, 1], ['Contains Gluten', 'Gluten-Free'], rotation=0) plt.show()

Run to view results

import matplotlib.pyplot as plt # Calculate Average Ingredient Length (Words) df['ingredients_length_words'] = df['ingredients'].apply(lambda x: len(x.split())) # Bar Plot of Average Ingredient Length (Words) average_length_words = df.groupby('contains_gluten')['ingredients_length_words'].mean() average_length_words.plot(kind='bar') plt.xlabel('Contains Gluten') plt.ylabel('Average Ingredient Length (Words)') plt.title('Average Ingredient List Length (Words)') plt.xticks([0, 1], ['Contains Gluten', 'Gluten-Free'], rotation=0) plt.show()

Run to view results

from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay # Predict Test Set Results y_pred = model.predict(x_test) # Compute Confusion Matrix cm = confusion_matrix(y_test, y_pred) # Display Confusion Matrix disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['No Gluten', 'Contains Gluten']) disp.plot(cmap='Blues') plt.title('Confusion Matrix of Logistic Regression Model') plt.show()

Run to view results

import matplotlib.pyplot as plt import seaborn as sns # Histogram of Ingredient Lengths plt.figure(figsize=(10, 6)) sns.histplot(data=df, x='ingredients_length_words', hue='contains_gluten', multiple='stack', bins=30) plt.xlabel('Ingredient Length (Words)') plt.ylabel('Frequency in 100,000 Samples') plt.title('Distribution of Ingredient Lengths') plt.legend(['Contains Gluten', 'Gluten-Free']) plt.show()

Run to view results