Data drifts pose a critical challenge in the lifecycle of machine learning (ML) models, affecting their performance and reliability. In response to this challenge, we present a microbenchmark study, called D3Bench, which evaluates the efficacy of open-source drift detection tools. D3Bench examines the capabilities of Evidently AI, NannyML, and Alibi-Detect, leveraging real-world data from two smart building use cases.We prioritize assessing the functional suitability of these tools to identify and analyze data drifts. Furthermore, we consider a comprehensive set of non-functional criteria, such as the integrability with ML pipelines, the adaptability to diverse data types, user-friendliness, computational efficiency, and resource demands. Our findings reveal that Evidently AI stands out for its general data drift detection, whereas NannyML excels at pinpointing the precise timing of shifts and evaluating their consequent effects on predictive accuracy.

The opportunity to be a part of the exciting early stages of a well-funded, European-based Open Source start-up that has massive growth and venture potential Fully Remote Working Environment 50€/month development budget to learn Data Science, Causal ML, Bayesian Inference or anything you like that applies to your role.

The opportunity to be a part of the exciting early stages of a well-funded, European-based Open Source start-up that has massive growth and venture potential Fully Remote Working Environment 50€/month development budget to learn Data Science, Causal ML, Bayesian Inference or anything you like that applies to your role.

Open Source Library for post deployment data science; estimating model performance, drift detection, model monitoring and other production-ready machine learning needs.