Introduction to Loss Functions - Algorithmia Blog

The loss function is the bread and butter of modern Machine Learning; it takes your algorithm from theoretical to practical and transforms neural networks from glorified matrix multiplication into Deep Learning. This post will explain the role of loss functions and how they work, while surveying a few of the most popular of the past decade. At its core, a loss function is incredibly simple: it's a method of evaluating how well your algorithm models your dataset. If your predictions are totally off, your loss function will output a higher number. As you change pieces of your algorithm to try and improve your model, your loss function will tell you if you're getting anywhere.