Scaling Machine Learning Models With Docker Swarm

There are alternatives to Docker Swarm for container orchestration, namely Kubernetes, which is also arguably the more popular option. There are also cloud-based solutions for deploying machine learning models (e.g. However there are times where we may want to roll our own solution, whether it is for learning or for self-hosting the entire pipeline (i.e., not relying on the cloud). The latter being quite common in government or regulated industries. In my job where data scientists and engineers often need to deploy within various sensitive and air-gapped networks that are "infra-constrained" (e.g., one cannot expect an on-premise Kubernetes cluster raring to go), we had to be self-sufficient in deploying and scaling ML models in a variety of constrained environments. Docker Swarm is my preferred choice in these admittedly niche scenarios.