A Guide to Scaling Machine Learning Models in Production

The workflow for building machine learning models often ends at the evaluation stage: you have achieved an acceptable accuracy, and "ta-da! Beyond that, it might just be sufficient to get those nice-looking graphs for your paper or for your internal documentation. In fact, going the extra mile to put your model into production is not always needed. And even when it is, this task is delegated to a system administrator. However, nowadays, many researchers/engineers find themselves responsible for handling the complete flow from conceiving the models to serving them to the outside world.