Are AI devices evaluated appropriately?

The researchers offered concrete evidence of that risk by conducting a case study of a deep learning model that analyzes chest X-rays for signs of collapsed lungs. The system was trained and tested on patient data from Stanford Health Center, but Zou and his colleagues tested it against patient data from two other sites -- the National Institute of Health in Bethesda, Md., and Beth Israel Deaconess Medical Center in Boston. Sure enough, the algorithms were almost 10 percent less accurate at the other sites. In Boston, moreover, they found that their accuracy was higher for white patients than for Black patients. AI systems have been famously vulnerable to built-in racial and gender biases, Zou notes.