Evaluation Metrics for Machine Learning Models

Machine learning as a field is full of technical terms, making it difficult for beginners to get started. One might see things like "deep learning," "the kernel trick," "regularization," "overfitting," "semi-supervised learning," "cross-validation," etc. But what in the world do they mean? One of the core tasks in building any machine learning model is to evaluate its performance. So how would one measure the success of a machine learning model?