How to run machine learning at scale -- without going broke

Machine learning is computationally expensive -- and because serving real-time predictions means running your ML models in the cloud, that computational expense translates into real dollars. Put another way, if you wanted to add a translation feature to your app that automatically translated text to your user's preferred language, you would deploy an NLP model as a web API for your app to consume. To host this API, you would need to deploy it through a cloud provider like AWS, put it behind a load balancer, and implement some kind of autoscaling functionality (probably involving Docker and Kubernetes). None of the above is free, and if you're dealing with a large amount of traffic, the total cost can get out of hand. This is especially true if you aren't optimizing your spend.