When we talk about Regression, we often end up discussing Linear and Logistics Regression. Do you know there are 7 types of Regressions? Linear and logistic regression is just the most loved members from the family of regressions. Last week, I saw a recorded talk at NYC Data Science Academy from Owen Zhang, current Kaggle rank 3 and Chief Product Officer at DataRobot. He said, 'if you are using regression without regularization, you have to be very special!'. I hope you get what a person of his stature referred to. I understood it very well and decided to explore regularization techniques in detail.

One of the major aspects of training your machine learning model is avoiding overfitting. The model will have a low accuracy if it is overfitting. This happens because your model is trying too hard to capture the noise in your training dataset. By noise we mean the data points that don't really represent the true properties of your data, but random chance. Learning such data points, makes your model more flexible, at the risk of overfitting.

Regression is an incredibly popular and common machine learning technique. Often the starting point in learning machine learning, linear regression is an intuitive algorithm for easy-to-understand problems. It can generally be used whenever you're trying to predict a continuous variable (a variable that can take any value in some numeric range), linear regressions and its relatives are often strong options, and are almost always the best place to start. This blog assumes a functional knowledge of ordinary least squares (OLS) linear regression. You can read more about OLS linear regression here, here, or here.