We live in an age of rapid AI innovation and progress. Yet even as academics and researchers make astonishing advancements, demonstrating real business value and positive return on investment is challenging. Developing cutting edge AI applications based on machine learning models integrated with existing business software is a common challenge. This article discusses a few of the core pain points and strategies to address them. The first challenge most organizations encounter is the increased complexity of preparing data and dataset management.

In every project, there is a phase where the business_specification is created that usually entails a timeframe, budget, and goal of the machine learning project. When say goal, I mean a set of KPIs, business metrics, or if you are super lucky machine learning metrics. At this stage, it is very important to manage business expectations but it's a story for another day. If you are interested in those things I suggest you take a look at some articles by Cassie Kozyrkov, for instance, this one. Assuming that you and your team know what is the business goal you can do initial_research and cook up a baseline approach, a first creative_idea.

When a data engineer is building a pipeline she can usually estimate how much time implementing a step in that pipeline would take. To a large extent, that information is available to her. When a data scientist is doing data exploration, the information needed to estimate the time to extract actionable insights is hidden in the very data that she is exploring. It is not available prior to the exploration. Not to mention, discussions with other stakeholders both technical and business side that arise because of the analysis which makes it even harder to estimate.