visual intuition
A Visual Intuition For Regularization in Deep Learning
In machine learning, regularization is a approach used to combat high variance -- in other words, the issue of your model learning to reproduce the data, rather than the underlying semantics about your problem. In an analogous way to humans learning, the idea is to construct your homework problems to test and build for knowledge, rather than simply rote learning: for example, learning multiplication tables as opposed to learning how to multiply. This kind of phenomenon is especially prevalent in learning by neural networks -- with great learning capacity comes a large likelihood for memorization, and it is up to us practitioners to guide deep learning models into soaking up our problem, not our data. Many of you will have come across these methods in the past, and may have developed your own intuition for how different regularization methods affect the outcome. For those of you who don't (and even for those who do!) this article provides a visual guide for how neural network parameters are shaped by regularization.