The Rise of the Model Servers

One of the exciting developments in machine learning recently is the rapid emergence of a new class of model servers. Model servers simplify the task of deploying machine learning at scale, the same way app servers simplify the task of delivering a web app or API to end users. The rise of model servers, coupled with increasingly interoperable models, will likely accelerate the adoption of user-facing machine learning in the wild. Although there has been an abundance of open source machine learning software, much of the ecosystem has been focused on model-building. The large Internet companies have built their own model serving infrastructure (such as FBLearner Predictor and Michelangelo), but there have been few easy options for the rest of us.