Machine Learning - Dzone Refcardz

To avoid an over-fitting problem (the trained model fits too well with the training data and is not generalized enough), the regularization technique is used to shrink the magnitude of Ɵi. This is done by adding a penalty (a function of the sum of Ɵi) into the cost function. In L2 regularization (also known as Ridge regression), Ɵi2 will be added to the cost function. In L1 regularization (also known as Lasso regression), Ɵi will be added to the cost function. Both L1, L2 will shrink the magnitude of Ɵi.