Reproducibility Challenges in Machine Learning for Health

Last year the United States Food and Drug Administration (FDA) cleared a total of 12 AI tools that use machine learning for health (ML4H) algorithms to inform medical diagnosis and treatment for patients. The tools are now allowed to be marketed, with millions of potential users in the US alone.Because ML4H tools directly affect human health, their development from experiments in labs to deployment in hospitals progresses under heavy scrutiny. A critical component of this process is reproducibility. A team of researchers from MIT, University of Toronto, New York University, and Evidation Health have proposed a number of "recommendations to data providers, academic publishers, and the ML4H research community in order to promote reproducible research moving forward" in their new paper Reproducibility in Machine Learning for Health. Just as boxers show their strength in the ring by getting up again after being knocked to the canvas, researchers test their strength in the arena of science by ensuring their work's reproducibility.