Model serving and deployment is one of the pillars of the MLOps stack. In this article, I'll dive into it and talk about what a basic, intermediate, and advanced setup for model serving look like. Let's start by covering some basics. Training a machine learning model may seem like a great accomplishment, but in practice, it's not even halfway from delivering business value. For a machine learning initiative to succeed, we need to deploy that model and ensure it meets our performance and reliability requirements. You may say, "But I can just pack it into a Docker image and be done with it". In some scenarios, that could indeed be enough. When people talk about productionizing ML models, they use the term serving rather than simply deployment. So what does this mean?

I know this has been asked many times and in many different ways. And there are tons of blog posts, articles, videos and courses addressing this and comparing hundreds of tools, libraries, frameworks… And that's part of my problem: I am facing so many options that I feel like Buridan's ass, dying of starvation for not knowing what to do. Although I don't want to write too much, I need to speak a little about our situation, in order to put the question in our context. We have only four people, which could be qualified as beginner data scientists. One of us has a profile that is a little bit more "engineer", so data engineer could be more suitable for him.

MLOps is a set of best practices that revolve around making machine learning in production more seamless. The purpose is to bridge the gap between experimentation and production with key principles to make machine learning reproducible, collaborative, and continuous. MLOps is not dependent on a single technology or platform. However, technologies play a significant role in practical implementations, similarly to how adopting Scrum often culminates in setting up and onboarding the whole team to e.g. To make it easier to consider what tools your organization could use to adopt MLOps, we've made a simple template that breaks down a machine learning workflow into components.