An end-to-end Machine Learning solution is an important way to bring AI to production and make it available for mass consumption and usage. But today, most AI practitioners simply do the pre-processing, training, evaluation and tuning stages and leave the remaining part to DevOps engineers. As such, a new field of development named MLOps has come into the mainstream. The focus has shifted from simply training and evaluation to also bringing and integrating it to production pipelines. On an individual level as well, knowing how to bring your model to the public is an important tool to have in an AI practitioner's skill-set.

FastAPI is a new Python API framework that is more and more used in production today. We are using FastAPI under the hood behind NLP Cloud. NLP Cloud is an API based on spaCy and HuggingFace transformers in order to propose Named Entity Recognition (NER), sentiment analysis, text classification, summarization, and much more. FastAPI helped us quickly build a fast and robust machine learning API serving NLP models. Let me tell you why we made such a choice, and show you how to implement an API based on FastAPI and spaCy for Named Entity Recognition (NER).

Flask is not a web server. It is a micro web application framework, a set of tools and libraries that make it easier and prettier to build web applications. Flask comes with Werkzeug, a WSGI utility library that provides a simple web server for development purposes. While Flask's development server is good enough to test the main functionality of the app, we shouldn't use it in production. While lightweight and easy to use, Flask's built-in server is not suitable for production as it doesn't scale well and by default serves only one request at a time.

At some point, we all have struggled in deploying our trained Machine Learning model and a lot of questions start popping up into our mind. What is the best way to deploy a ML model? How do I serve the model's predictions? Which server should I use? Should I use flask or django for creating REST API? Don't worry, I got you covered with all of it!!:) In this tutorial, we will learn how to train and deploy a machine learning model in production with more focus on deployment because this is where we all data scientists get stuck.

First we'll need to write a function that can take an unclassified entry and perform a prediction on it. To do this, the script will need to rebuild the model in memory based on the pickle file (model.dat, in this case), and feed it a new entry to allow it to make a prediction. While it's possible to retrain a model from scratch each time we want to make a prediction, this is incredibly resource intensive (especially in larger examples) and is a fundamentally different process from making a standalone inference, and as such, is very bad practice in machine learning. I've written a predict function within a new file, predict.py(below), For this prediction, the model requires 4 numerical inputs (sepal_length, sepal_width, petal_length, petal_width -- in this order) and returns a class prediction containing one of three species (Iris-setosa, Iris-versicolor, Iris-virginica).

We've built a state-of-the-art, self-healing Artificial Intelligence cluster using the public cloud at AWS using cutting edge technology. We can't reveal our secret sauce (otherwise it wouldn't be secret anymore!) but read on to learn about how our Docker expert and Senior Software Engineer Jett Jones and the Bonsai Seattle team built a scalable, resilient AI Engine using some of today's most exciting configuration management, monitoring, and networking technologies. Provisioning resources in AWS is a tedious and error-prone task, due to the enormous number of possible options. To make this process repeatable there are several options, but the notion of a "tainted" machine drew us to HashiCorp's Terraform for machine-level provisioning over similar solutions such as Cloud Formation for launching instances and creating network configurations. The recently released Docker Swarm manages post-provisioning creation of services and some of the IPC functionality that allows the services to work together, as well as ensuring that services are launched on the correct machines and providing process management features around starting, stopping, and failure recovery.