The 5 Components Towards Building Production-Ready Machine Learning System

The biggest issue facing machine learning is how to put the system into production. To conceptualize this framework, there is a significant paper from Google called ML Test Score -- A Rubric for Production Readiness and Technical Debt Reduction -- which is an exhaustive framework/checklist from practitioners at Google. It is a follow-up to previous work from Google, such as (1) Hidden Technical Debt in ML Systems, (2) ML: The High-Interest Credit Card of Technical Debt, and (3) Rules of ML: Best Practices for ML Engineering. As seen in Figure 1 from the paper above, ML system testing is more complex a challenge than testing manually coded systems, since ML system behavior depends strongly on data and models that cannot be sharply specified a priori. One way to see this is to consider ML training as analogous to the compilation, where the source is both code and training data.