Understanding Maximum Likelihood Estimation in Supervised Learning

We will understand how our assumptions on the data enable us to create meaningful optimization problems. In fact, we will derive commonly used criteria such as cross-entropy in classification and mean square error in regression. Finally, I am trying to answer an interview question that I encountered: What would happen if we use MSE on binary classification? To begin, let's start with a fundamental question: what is the difference between likelihood and probability? The data xxx are connected to the possible models θ\thetaθ by means of a probability P(x,θ)P(x,\theta)P(x,θ) or a probability density function (pdf) p(x,θ)p(x,\theta)p(x,θ).