Mathematical foundation for Noise, Bias and Variance in #NeuralNetworks – Autonomous Agents -- #AI

Neural Nets are quite powerful models in machine learning which are used for learning the behavior of high dimensional data. Typically, data is not always present in its purest form, the signal. Most of the data that is made available for machines during training or runtime prediction do have some amount of noise. During training, the Neural Nets can get highly sensitive to the noise in the input and can start overfitting the learning to the noise in the input. In the previous post titled "Algorithms to Improve NeuralNetwork Accuracy", we learnt about the overfitting problem in Neural Nets and how to break it using L1/L2 regularizers, Weight penalties decay and constraints.