The Interplay Between AI and Business Objectives

"The best way to have a good idea is to have a lot of ideas." Let's assume we are running an e-commerce search engine that uses machine learning on user-issued queries to identify the intended product category. Say the model in production incurs a 20ms prediction latency and has 90% accuracy. A natural next goal from a modeling perspective would be to drive the accuracy higher, say to 95% or beyond. However, we know that improving the accuracy almost always requires the consumption of more computational resources for training models and may also increase the inference latency.