By Sandeep Uttamchandani, Ph.D., Both a Product/Software Builder (VP of Engg) & Leader in operating enterprise-wide Data/AI initiatives (CDO) In this article, I share the ten deadly sins during ML model training -- these are the most common as well as the easiest to overlook. During model training, there are scenarios when the loss-epoch graph keeps bouncing around and does not seem to converge irrespective of the number of epochs. There is no silver bullet as there are multiple root causes to investigate -- bad training examples, missing truths, changing data distributions, too high a learning rate. The most common one I have seen is bad training examples related to a combination of anomalous data and incorrect labels. The more the same data is used for parameter and hyperparameter settings, the lesser confidence that the results will actually generalize.
The most decisive criterion appears at the top of the plot. We hope that Shapash will be useful in building trust in AI. Thank you in advance to all those who will give us their feedback, idea… Shapash is opensource! Feel free to contribute by commenting on this post or directly on the GitHub discussions.
When solving machine learning problems, simply training a model based on a problem-specific training machine learning algorithm does not guarantee either that the resulting model fully captures the underlying concept hidden in the training data or that the optimum parameter values were chosen for model training. Failing to test a model's performance means an underperforming model could be deployed on the production system, resulting in incorrect predictions. Choosing one model from the many available options based on intuition alone is risky. By generating different metrics, the efficacy of the model can be assessed. Use of these metrics reveals how well the model fits the data on which it was trained.
Companies can engage in different approaches to model development. From fully managed ML services, all the way to custom models. Depending on business requirements, available expertise, and planning constraints, they must make a choice: should they develop custom solutions from scratch? Or should they choose an off-the-shelf service? For all stages of ML workloads, a decision must be met concerning how the different puzzle pieces will fit together.
Do you understand how your machine learning model works? Despite the ever-increasing usage of machine learning (ML) and deep learning (DL) techniques, the majority of companies say they can't explain the decisions of their ML algorithms . This is, at least in part, due to the increasing complexity of both the data and models used. It's not easy to find a nice, stable aggregation over 100 decision trees in a random forest to say which features were most important or how the model came to the conclusion it did. This problem grows even more complex in application domains such as computer vision (CV) or natural language processing (NLP), where we no longer have the same high-level, understandable features to help us understand the model's failures.
AI systems are becoming increasingly popular and central in many industries. They decide who might get a loan from the bank, whether an individual should be convicted, and we may even entrust them with our lives when using systems such as autonomous vehicles in the near future. Thus, there is a growing need for mechanisms to harness and control these systems so that we may ensure that they behave as desired. One important issue that has been gaining popularity in the last few years is fairness. While usually ML models are evaluated based on metrics such as accuracy, the idea of fairness is that we must ensure that our models are unbiased with regard to attributes such as gender, race and other selected attributes.
For data scientists, a key part of interpreting machine learning models is understanding which factors impact predictions. In order to effectively use machine learning in their decision-making processes, companies need to know which factors are most important. For example, if a company wants to predict the likelihood of customer churn, it might also want to know what exactly drives a customer to leave a company. In this example, the model might indicate that customers who purchase products that rarely go on sale are much more likely to stop purchasing. Armed with this knowledge, a company can make smarter pricing decisions in the future.
The world and data are not static. But most machine learning models are. Once they are in production, they become less relevant with time. The data distributions evolve, the behavioral patterns change, and models need updates to keep up with new reality. The usual process is to retrain the models at defined intervals.
Corporations are going through rough times. The times are uncertain, and having to make customer experiences more and more seamless and immersive isn't taking off any of the pressure on companies. In that light, it's understandable that they're pouring billions of dollars into the development of machine learning models to improve their products. Companies can't just throw money at data scientists and machine learning engineers, and hope that magic happens. Here's how AI can improve your company's customer journey The data speaks for itself.