What "no free lunch" really means in machine learning

You don't have to cook or spend any of your hard-earned money. The truth is unless if you count special talks and lectures in graduate school that promise free pizza, there is no free lunch in machine learning. The "no free lunch" (NFL) theorem for supervised machine learning is a theorem that essentially implies that no single machine learning algorithm is universally the best-performing algorithm for all problems. This is a concept that I explored in my previous article about the limitations of XGBoost, an algorithm that has gained immense popularity over the last five years due to its performance in academic studies and machine learning competitions. The goal of this article is to take this often misunderstood theorem and explain it so that you can appreciate the theory behind this theorem and understand the practical implications that it has on your work as a machine learning practitioner or data scientist.