Training and Deployment Pipeline, Part 1

To remind you with a visual, here, in figure 1, is the whole pipeline. I've circled the part of the system we'll address in this article. You may ask, what exactly is a pipeline and why do we use one, whether for ML production or any programmatic production operation which is managed by orchestration? You typically use pipelines when the job, such as training or other operation handled by orchestration, has multiple steps that occur in sequential order: do step A, do step B, and like this. Putting the steps into a ML production pipeline has multiple benefits.