IoT ML

Multi-Label Classification Using Machine Learning for IoT Application

Problem

Smart thermostats are a standard device in most houses now. To set the desired temperature, the thermostat must know if the occupants are home, not home, or sleeping. Most thermostats come equipped with the following sensors:

Sound level sensor

Light level sensor

Vibration sensor

Real-time clock (Time of Day)

Based on this data a pattern recognition algorithm can be used to predict whether the occupants are at home, sleeping, or not at home. This notebook shows the implementation of a simple Neural network to perform this task.

# Import Libraries import pandas as pd import matplotlib.pyplot as plt import numpy as np from keras.layers import Dense, Dropout from keras.models import Sequential from keras.callbacks import EarlyStopping from sklearn.preprocessing import OneHotEncoder from keras.layers import BatchNormalization from sklearn.model_selection import train_test_split ! pip install openpyxl

Importing Data

The dataset is stored in an excel file and contains multiple rows of data with 4 input features and 1 output feature (Status). A snippet of the data is shown below.

# Read the data from the file df = pd.read_excel('IoT_Dataset.xlsx', sheet_name='Training set')

Exploring Data

The first step before developing a model is to explore the dataset and based on that information an appropriate model can be selected.

# Print out the first six rows of the dataset print(df.head(6))

# Statistical description of the data in each column print(df.describe())

# Display the type of data in each column df.dtypes

# Check if there are any null values in the dataset print(df.isnull().values.any())

Model Selection

Given that the input data is numerical and quite structured, a good model for predicting the output could be a vanilla neural network. The network would have 4 numerical input features and one output class prediction. The problem can be described as a multi-class classification problem.

Data Preparation

In order to prepare the data for training, the input columns will have to be converted to floats. Additionally, the categorical labels will have to be one hot encoded, so that a softmax layer can be used at the output layer of the network.

# Convert the data into numpy arrays train = df.to_numpy() # The first 4 columns are the features and the last column is the label # Split the data into features and labels for training # One hot encode the labels x = train[:,0:4].astype('float32') encoder = OneHotEncoder(sparse=False) y = encoder.fit_transform(train[:,4].reshape(-1,1))

Training a Model

The first step before training is to split the data into training and validation sets. The training set will be used to train the model (find the optimized weights of the NN) and the validation set will be used to save the weights with the highest accuracy. 70% of the data set for training and 30% for validation. Due to the low number of data points, a separate set is not used for testing.

# Split the data into training and validation sets x_train, x_valid, y_train, y_valid = train_test_split(x, y, test_size=0.30)

Once the data is split, a neural network can be created using Keras by sequentially adding layers. The designed neural network contains the following layers.

Batch Normalization: Batch normalization applies a transformation that maintains the mean output close to 0 and the output standard deviation close to 1.

Input Layer: The input layer contains four inputs which are the 4 features from the dataset.

Hidden Layers: This NN contains three hidden layers with 100 neurons each with relu activation function applied to the output of each neuron.

Dropout Layer: The Dropout layer randomly sets input units to 0 with a frequency of rate (0.2 in this network) at each step during training time, which helps prevent overfitting. Inputs not set to 0 are scaled up by 1/(1 - rate) such that the sum over all inputs is unchanged.

Callback: Callback in this network is used to perform early stopping. In this case, the model is stopped to maximize validation accuracy.

model = Sequential() model.add(BatchNormalization()) model.add(Dense(100, input_dim=4, activation='relu')) model.add(Dense(100, activation='relu')) model.add(Dropout(0.2)) model.add(Dense(100, activation='relu')) model.add(Dense(3, activation='softmax')) model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) callback = EarlyStopping( monitor='val_accuracy', min_delta=0, patience=50, verbose=1, mode='auto', baseline=None, restore_best_weights=True ) history = model.fit(x_train, y_train, validation_data = (x_valid,y_valid),epochs = 200, batch_size = 10, verbose = 1,callbacks=[callback])

plt.plot(history.history['loss'], label='Training Loss') plt.plot(history.history['val_loss'], label = 'Validation Loss') plt.legend() plt.xlabel('Epochs') plt.ylabel('Mean Squared Error') plt.show()

Using Model for Prediction

Once the model is created, it can be used to make predictions on a dataset that does not contain any labels. A separate data file is loaded below that does not contain any labels.

# Predict the labels for inference prediction_data = pd.read_excel('IoT_Dataset.xlsx', sheet_name='Test data') print(prediction_data.head(3))

The dataset is converted into a NumPy array and the Keras predict function is used to make a prediction. The output of the neural network passes through a softmax function which converts a vector of values to a probability distribution. The elements of the output vector are in the range (0, 1) and sum to 1.

prediction_data = prediction_data.to_numpy() x_prediction = prediction_data[:,0:4].astype('float32') # Predict the output y_prediction = model.predict(x_prediction,verbose=0) print(y_prediction)

The arg max function and the label mapping can be used to essentially decode the one hot encoded output of the NN back to their corresponding classes as shown below.

# Map of labels to their corresponding classes label_map = {0:'Home', 1:'Not Home', 2:'Sleeping'} label = [label_map[i] for i in np.argmax(y_prediction, axis=1)]

This produces a list of labels for each data point in the unseen dataset.

print(label)

.css-15w88e5{color:var(--chakra-colors-fg-neutral-primary);font-weight:inherit;letter-spacing:-0.09px;}Multi-Label Classification Using Machine Learning for IoT Application