Avoiding Complexity of Machine Learning Systems

Before even thinking about complexity in your ML system, ask yourself if your product feature actually needs an ML solution. Sometimes, ML adds complexity to your system when you could just use a simpler heuristic algorithm that does not require feature engineering, model tuning, continuous training, or model deployment. However, when there are already ML models built for other purposes which you can reuse, going with a heuristic adds complexity. A quick and dirty heuristic might seem like a short-term gain, but is really a long-term pain. Over time it becomes increasingly difficult to understand, depend on, and maintain all the ad-hoc heuristics. The product can also suffer when there are too many different ways to do similar things, resulting in inconsistent user-facing behavior.