Intro to Regularization With Ridge And Lasso Regression with Sklearn

Ordinary Least Squares is one of the easiest and most widely used ML algorithms. But it suffers from a fatal flaw -- it is super easy for the algorithm to overfit the training data. But as the number of predictor variables (or dimensions) increases, the coefficients β_i also tend to get very large. With large coefficients, it is easy to predict nearly everything -- you just take the relevant combination of individual slopes (βs) and you get the answer. That's why it is common for linear regression models to overfit the training data.