Understanding and comparing Batch Norm with all different variations

In this article we explain what is batch normalization (BN) and how all it variations works. We try to explain the difference conceptually and mathematically. At the end you will have a deeper understanding on how normalization it's done in modern neural network. Batch Normalization is a widely used method in deep learning to make training faster and more stable. The main idea is to normalize each values of each internal layer of a deep neural network.