It first introduces an example using Flask to set up an endpoint with Python, and then shows some of issues to work around when building a Keras endpoint for predictions with Flask.

The Keras library provides an approachable interface to deep learning, making neural networks accessible to a broad audience. However, one of the challenges I've faced is transitioning from exploring models in Keras to productizing models. Keras is written in Python, and until recently had limited support outside of these languages. While tools such as Flask, PySpark, and Cloud ML make it possible to productize these models directly in Python, I usually prefer Java for deploying models. Projects such as ONNX are moving towards standardization of deep learning, but the runtimes that support these formats are still limited.

This post demonstrates how to set up an endpoint to serve predictions using a deep learning model built with Keras. It first introduces an example using Flask to set up an endpoint with Python, and then shows some of issues to work around when building a Keras endpoint for predictions with Flask. The goal of this post is to show how to set up a Keras model as an endpoint on an EC2 instance with AWS. Some of the issues that I'll cover include handling a custom metric when using model persistence with Keras, dealing with multi-threading concerns when using Keras in combination with Flask, and getting it all running on an EC2 instance. The complete code listing for this post is available on GitHub.