Like many ML practitioners, I started my ML journey with Kaggle competitions. But the comfortable setup of Kaggle, where you are handed largely clean data along with features and labels, could not be further from the reality of today's ML practitioner. In fact, the model building process itself is merely a small fraction of the work that needs to be done when developing an ML solution and deploying and maintaining it in production. It is useful to speak about ML production systems in terms of various degrees of maturity, where the least mature systems are one-off models, and the most mature systems run on autopilot, updating themselves with minimal human intervention. Here, I make a broad categorization of ML systems into four levels of increasing maturity, and discuss some of the challenges involved at each level. Disclaimer: given the choice of medium (a blog post, not a book chapter), this list will certainly be incomplete, and I didn't intend it to be.

Like many ML practitioners, I started my ML journey with Kaggle competitions. But the comfortable setup of Kaggle, where you are handed largely clean data along with features and labels, could not be further from the reality of today's ML practitioner. In fact, the model building process itself is merely a small fraction of the work that needs to be done when developing an ML solution and deploying and maintaining it in production. It is useful to speak about ML production systems in terms of various degrees of maturity, where the least mature systems are one-off models, and the most mature systems run on autopilot, updating themselves with minimal human intervention. Here, I make a broad categorization of ML systems into four levels of increasing maturity, and discuss some of the challenges involved at each level.

The simplest approach is to run the ML algorithm locally to compute predictions on prepared test data and share predictions with others. This approach is easy and fast in implementation. However, it has many drawbacks. It is hard to govern, monitor, scale and collaborate. The second, similar approach, is to hard-code the ML algorithm in the system's code.