ml-notes-why-the-log-likelihood-24f7b6c40f83?utm_content=buffer04f75&gi=96bc516d2eed

Secretly, you are hoping that your model will predict future experiences, people call that "generalisation". If we had a sum instead of a product, we could load one datum at a time compute its partial derivatives, accumulating those gradients and apply the optimisation at the end. This little term is what people call the regularisation term, it takes into account your "prior" knowledge of the problem. Notice how engineering problems pushed us to find better notations or better optimisation procedures, surprisingly in machine learning, the basic probability theories are often not that complicated to grasp but the engineering feat to make them actually work are insane.