Energy Preserving Neural Networks – ML Review – Medium

In the current deep learning era, there is a neural network architecture available for possibly every problem; for instance, ResNet / CNN for computer vision problems, RNN with attention for Language / Speech problems and so on. The Deep Learning models have surpassed almost all the traditional Machine Learning techniques and are the current SOTA for various problems that were deemed to be impossible to achieve for a computer. However, if we take a deep look at the primary computations performed by a neural network, we can obtain better insights about the intricate details of its working. One prominent phenomenon that I observed is that the norm of the data that we process through the network changes to a certain extent. In order to address this problem, the use of batch-normalization layer is common.