In this paper, I train a Bidirectional Long Short-Term Memory (BiLSTM) model on a novel dataset of voter registration data from all 50 US states and create an ensemble that achieves up to 36.8% higher out of sample (OOS) F1 scores than the best performing machine learning models in the literature. Additionally, I construct the most comprehensive database of first and surname distributions in the US in order to improve the coverage and accuracy of Bayesian Improved Surname Geocoding (BISG) and Bayesian Improved Firstname Surname Geocoding (BIFSG). Finally, I provide the first high-quality benchmark dataset in order to fairly compare existing models and aid future model developers.

Clinical notes are becoming an increasingly important data source for machine learning (ML) applications in healthcare. Prior research has shown that deploying ML models can perpetuate existing biases against racial minorities, as bias can be implicitly embedded in data. In this study, we investigate the level of implicit race information available to ML models and human experts and the implications of model-detectable differences in clinical notes. Our work makes three key contributions. First, we find that models can identify patient self-reported race from clinical notes even when the notes are stripped of explicit indicators of race. Second, we determine that human experts are not able to accurately predict patient race from the same redacted clinical notes. Finally, we demonstrate the potential harm of this implicit information in a simulation study, and show that models trained on these race-redacted clinical notes can still perpetuate existing biases in clinical treatment decisions.

Artificial intelligence (AI) is used in a wide variety of health care settings, from analyzing medical images to assisting with surgical procedures. While AI can sometimes outperform trained clinicians, these superhuman abilities are not always fully understood. In a recent study published in The Lancet Digital Health, researchers found that AI models could accurately predict self-reported race in several different types of radiographic images--a task not possible for human experts. These findings suggest that race information could be unknowingly incorporated into image analysis models, which could potentially exacerbate racial disparities in the medical setting. "AI has immense potential to revolutionize the diagnosis, treatment, and monitoring of numerous diseases and conditions and could dramatically shape the way that we approach health care," said first study author and NIBIB Data and Technology Advancement (DATA) National Service Scholar Judy Gichoya, M.D. "However, for AI to truly benefit all patients, we need a better understanding of how these algorithms make their decisions to prevent unintended biases."

As more machine learning tools reach patients, developers are starting to get smart about the potential for bias to seep in. But a growing body of research aims to emphasize that even carefully trained models -- ones built to ignore race -- can breed inequity in care. Researchers at the Massachusetts Institute of Technology and IBM Research recently showed that algorithms based on clinical notes -- the free-form text providers jot down during patient visits -- could predict the self-identified race of a patient, even when the data had been stripped of explicit mentions of race. It's a clear sign of a big problem: Race is so deeply embedded in clinical information that straightforward approaches like race redaction won't cut it when it comes to making sure algorithms aren't biased. "People have this misconception that if they just include race as a variable or don't include race as variable, it's enough to deem a model to be fair or unfair," said Suchi Saria, director of the machine learning and health care lab at Johns Hopkins University and CEO of Bayesian Health.

The miseducation of algorithms is a critical problem; when artificial intelligence mirrors unconscious thoughts, racism, and biases of the humans who generated these algorithms, it can lead to serious harm. Computer programs, for example, have wrongly flagged Black defendants as twice as likely to reoffend as someone who's white. When an AI used cost as a proxy for health needs, it falsely named Black patients as healthier than equally sick white ones, as less money was spent on them. Even AI used to write a play relied on using harmful stereotypes for casting. Removing sensitive features from the data seems like a viable tweak.

Artificial intelligence (AI) is used by medical facilities to help analyze x-rays and other medical scans, but a new study finds the technology can see more than just a patient's health – it can determine their race with startling accuracy. The study's 20 authors found deep learning models can identify race in chest and hand x-rays and mammograms among patients who identified as black, white and Asian. The algorithms correctly identify which images were from a black person more than 90 percent of the time, but also showed it was able to identity race with 99 percent accuracy at times. However, what is even more alarming is that the team was unable to explain how the AI systems were making accurate predictions, some of which were done with scans that were blurry or low-resolution. 'We emphasize that model ability to predict self-reported race is itself not the issue of importance,' Ritu Banerjee, associate professor of pediatrics at Washington University School of Medicine and lead author of the study, and collages wrote in the study published in arXiv.