Goto

Collaborating Authors

Designing a Python interface for machine learning engineering

#artificialintelligence

In order to do machine learning engineering, a model must first be deployed, in most cases as a prediction API. In order to make this API work in production, model serving infrastructure must first be built. This includes load balancing, scaling, monitoring, updating, and much more. At first glance, all of this work seems familiar. Web developers and DevOps engineers have been automating microservice infrastructure for years now.


Designing a Python interface for machine learning engineering

#artificialintelligence

In order to do machine learning engineering, a model must first be deployed, in most cases as a prediction API. In order to make this API work in production, model serving infrastructure must first be built. This includes load balancing, scaling, monitoring, updating, and much more. At first glance, all of this work seems familiar. Web developers and DevOps engineers have been automating microservice infrastructure for years now.


Deploy machine learning models in production

#artificialintelligence

Cortex is an open source machine learning deployment platform that makes it simple to deploy your machine learning models as web APIs on AWS. It combines TensorFlow Serving, ONNX Runtime, and Flask into a single tool that takes models from S3 and deploys them as web APIs. It also uses Docker and Kubernetes behind the scenes to autoscale, run rolling updates, and support CPU and GPU inference. The project is maintained by a venture-backed team of infrastructure engineers with backgrounds from Google, Illumio, and Berkeley. Minimal declarative configuration: Deployments can be defined in a single cortex.yaml


Serve Your ML Models in AWS Using Python

#artificialintelligence

Automate your ML model train-deploy cycle, garbage collection, and rollbacks, all from Python with an open-source PyPi package based on Cortex. It all started with modernization of a product categorization project. The goal was to replace complex low-level Docker commands with a very simple and user-friendly deployment utility called Cortex. The solution in the form of a Python package proved to be re-usable since we successfully used it as part of our recommendation engine project. We plan to deploy all ML projects like this. Since GLAMI relies heavily on open-source software, we wanted to contribute back and decided to open-source the package, calling it Cortex Serving Client.


cortexlabs/cortex

#artificialintelligence

Cortex is an open source platform that takes machine learning models--trained with nearly any framework--and turns them into production web APIs in one command. Autoscaling: Cortex automatically scales APIs to handle production workloads. Multi framework: Cortex supports TensorFlow, PyTorch, scikit-learn, XGBoost, and more. CPU / GPU support: Cortex can run inference on CPU or GPU infrastructure. Rolling updates: Cortex updates deployed APIs without any downtime.