Five steps to optimize AI deployments

With many organisations focusing research on enhancing AI capabilities, we see a lot of action and breakthroughs in the world of deep learning. Most state-of-the-art models are being open-sourced on TFHub, HuggingFace & PyTorch for everyone to experiment and enjoy. Yet, most of these models are heavy and not much useful for CPU based inference and deployments. We improved our inference speed and reduced the size of our docker image from 9.7 Gb to 3.7 Gb (3x reduction), which serves as the motivation for this article. In this article, we would like to present some of the best practices and tricks we followed deploying deep learning models on the cloud using Docker.