Scaling Machine Learning Applications - DATAVERSITY

When the number of users for a predictive model grows, it is expected (albeit often wrongly) that the machine learning powered systems will automatically scale to keep up with this growth. If the system fails to scale, processing requirements may outpace performance. Using an example from a LinkedIn article, a sample recommender system fails to recommend the desired list of products or services in a timely manner, which means the customer does not receive the product or service recommendations at the time of purchase. Though developing a scalable system can pose a serious challenge, shying away from building a scalable system can become a bigger problem and can result in lost customers or unrealized revenue. During scaling, many technical problems like workload issues, memory representation, framework restrictions, resource use vs. performance, and others can surface and stall the production.