Scaling Machine Learning from 0 to millions of users -- part 2

In part 1, we broke out of the laptop, and decided to deploy our prediction service on a virtual machine. By doing so, we discussed a few simple techniques that helped with initial scalability… and hopefully with reducing manual ops. Since then, despite a few production hiccups due the lack of high availability, life has been pretty good. However, traffic soon starts to increase, data piles up, more models need to be trained, etc. Technical and business stakes are getting higher, and let's face it, the current architecture will go underwater soon. Yes, it can be a short-term solution to use a large server for training and prediction.