Machine Learning 103: Loss Functions

In two previous articles I covered two of the most basic models used in machine learning -- linear regression and logistic regression. In both cases, we were interested in searching for the set of model parameters m that result in the best model predictions d' of the observed targets d, and in both cases this was done by minimizing some loss function L(m), which measures the error between d' and d. A good proportion of machine learning -- from simple linear regression to deep learning models, essentially involves the minimization of some sort of loss function -- and yet, many data science or machine learning books/tutorials/materials tend to place more emphasis on the model itself than on the loss function! In this article, we will continue on where we left off from the previous two articles and focus on loss functions before exploring more advanced models in future articles! Now, just as "best" is a very subjective word, so are loss functions!