Real-World Machine Learning: Model Evaluation & Optimization

The primary goal of supervised machine learning is accurate prediction. We want our ML model to be as accurate as possible when predicting on new data (for which the target variable is unknown). Said in a different way, we want our models, which have been built from some training data, to generalize well to new data. That way, when we deploy the model in production, we can be assured that the predictions generated are of high quality. Therefore, when we evaluate the performance of a model, we want to determine how well that model will perform on new data.