Question 2

2.1)

From the summary we can see that our model has four layers. One input layer (flatten) two hidden layers and one output layer that represents the numbers 0-9. It also shows how many neurons each layer has: the first one has 784 since it represents all the pixels in the 28 x 28 original image. The two hidden layers have 64 neurons each and the output layer has 10 neurons, one for each number. The number of neurons in the hidden layers can vary based what generates the optimal result. We also see that the toatal numbers of parameters are 55 050.

Most layers use the Rectified Linear Unit (ReLu) activation funciton. The ReLu is commonly used in neural such as this one because it computationally efficient becuase it doesnt use any advanced mathematical formula making it faster than other methods. Furthermore, it prevents a problem with vanishing gradients, for example when the sigmoid funciton gets close to 0 or 1. This could make it hard for model to learn when the gradient becomes small. The outout function uses softmax as activation. This, as opposed to the ReLu activation, creates a probalility distribution over the classification that together sum up to one. It then activates the node with the highest probability which is our output.

2.2)

The loss function used is categorical crossentropy. This is commonly used for these kinds of multidimensional where each sample only belong to one class. The model measures the dissimilarity between probability distributions and helps the model learn to predict the correct class labels. The formula is calculated as the negative sum over all classes i, from 1 to the total number of classes, of the product of the true label for that class and the logarithm of the predicted probability for that class.

2.3)

2.4)

Comment on 2.4)

A max accuracy of 0.9822 was achieved. This was not the same as Hiltons result of 0.9847. The reason behind this could be the use of different epochs and regularization factors. There is no mention of the number of epochs Hinton used and higher number of epoch could, up to a certain point, result in a higher accuracy. The case could also be that a higher epoch than Hilton was used and thus overfitted the model, resulting in a lower accuracy. Either way, the diffrence in accuracy may be a cause of different number of epochs.

The regularization factor also have an effect on the accuracy. This was seen above where different factors generated different accuracy. Since Hilton only said to use five factors inside an interval, the factors picked could differ.

Question 3

3.1)

Comment on 3.1)

When I first tried only using convolution I ended up with an accuracy of 0.986. Then I saw from the video that it is common to use MaxPooling together with convolutional layers. The MaxPooling divides the feature maps into smller subplots which makes the model more cumpotunally efficient and reduces the risk of overfitting. I then tested with MaxPooling and some different numbers of neurons and epochs and eventually got an accuracy of 0.9911 which is above 99 %

Video on MaxPooling2D: https://www.youtube.com/watch?v=WvoLTXIjBYU&list=PLQVvvaa0QuDfhTox0AjmQ6tvTgMBZBEXN&index=4&ab_channel=sentdex

3.2)

Convolutional layers and fully connected layers serve different purposes in machine learning models. Convolutional layers are designed to work better on images whereas fully connected layers works with every datatype.

Since the convolutional layers have fewer parameters which makes them less prone to overfittning, it is also better at detecting objects in images. For example a circle or line that makes up a number. The fully connected layer on the other hand are more efficient when it comes to data that is not structured as a imgaes with grids. This could be predicting the score of a game based on a set of features.

Since this task was an image classification task, convolutional layers worked better which also the report showed.