Digit Classification with Single-Layer Perceptron

Generally the first thought that comes to mind when one is about to apply Supervised Learning techniques on images is to make use of Convolutional Neural Networks (CNNs). Indeed, this type of neural network is the most suitable for this type of tasks, mainly due to the reduction of dimensionality. If we imagine a dataset of images where the images have been flattened (for example, an image that is a 4x4 matrix is converted to a 16-dimensional vector, as shown in Figure 1), the images are data points in an n-dimensional space, where n is the number of pixels in the image. As can be deduced, the dimensionality of the data when we talk about images is enormous, and therefore this implies having an immense number of parameters in the neural network, which in turn leads to a higher computational cost and execution time. CNNs reduce the dimensionality of the image in each layer of the neural network, also reducing the number of parameters required in training and optimizing the performance of the model for this type of tasks.