Simplified Math behind Dropout in Deep Learning

In this post, our objective is to understand the Math behind Dropout. However, before we get to the Math, let's take a step back and understand what changed with Dropout. This will be a motivation to touch the Math. Before Dropout, a major research area was regularization. Introduction of regularization methods in neural networks, such as L1 and L2 weight penalties, started from the early 2000s [1].