The four maturity levels of ML production systems

Like many ML practitioners, I started my ML journey with Kaggle competitions. But the comfortable setup of Kaggle, where you are handed largely clean data along with features and labels, could not be further from the reality of today's ML practitioner. In fact, the model building process itself is merely a small fraction of the work that needs to be done when developing an ML solution and deploying and maintaining it in production. It is useful to speak about ML production systems in terms of various degrees of maturity, where the least mature systems are one-off models, and the most mature systems run on autopilot, updating themselves with minimal human intervention. Here, I make a broad categorization of ML systems into four levels of increasing maturity, and discuss some of the challenges involved at each level.