Optimization in Machine Learning -- A Beginner's Guide

Before delving into optimization methods, it's critical to understand the various types of functions utilised in machine learning. Constrained functions are mathematical expressions that are subject to certain constraints or rules. These restrictions can take the form of equalities or inequalities that the function's input and output must satisfy. A limitation could, for example, be that the function's input must be inside a certain range. The two most prevalent types of constrained functions are Linear programming, in which both the objective function and constraints are linear, and quadratic programming, in which the objective function is quadratic in form.