Under the Hood of Deep Learning

The previous image is a simple architecture for a deep neural network. The goal of this post is to understand deep learning details and build your own network, rather than use the existing models as a black box! In this post, we will go over a simple neural network that can learn to recognize hand-written digits (MNIST dataset). Currently, there are various types of neural networks, but for the sake of simplicity, we will start with the vanilla form (aka "Multilayer Perceptron"). Please note that the circles in the previous diagrams called neurons.