Designing a Python interface for machine learning engineering

In order to do machine learning engineering, a model must first be deployed, in most cases as a prediction API. In order to make this API work in production, model serving infrastructure must first be built. This includes load balancing, scaling, monitoring, updating, and much more. At first glance, all of this work seems familiar. Web developers and DevOps engineers have been automating microservice infrastructure for years now.