How to Solve the Model Serving Component of the MLOps Stack - neptune.ai

Model serving and deployment is one of the pillars of the MLOps stack. In this article, I'll dive into it and talk about what a basic, intermediate, and advanced setup for model serving look like. Let's start by covering some basics. Training a machine learning model may seem like a great accomplishment, but in practice, it's not even halfway from delivering business value. For a machine learning initiative to succeed, we need to deploy that model and ensure it meets our performance and reliability requirements. You may say, "But I can just pack it into a Docker image and be done with it". In some scenarios, that could indeed be enough. When people talk about productionizing ML models, they use the term serving rather than simply deployment. So what does this mean?