Popular artificial intelligence tools are becoming more covertly racist as they advance, says an alarming new report. A team of technology and linguistics researchers revealed this week that large language models like OpenAI's ChatGPT and Google's Gemini hold racist stereotypes about speakers of African American Vernacular English, or AAVE, an English dialect created and spoken by Black Americans. "We know that these technologies are really commonly used by companies to do tasks like screening job applicants," said Valentin Hoffman, a researcher at the Allen Institute for Artificial Intelligence and co-author of the recent paper, published this week in arXiv, an open-access research archive from Cornell University. Hoffman explained that previously researchers "only really looked at what overt racial biases these technologies might hold" and never "examined how these AI systems react to less overt markers of race, like dialect differences". Black people who use AAVE in speech, the paper says, "are known to experience racial discrimination in a wide range of contexts, including education, employment, housing, and legal outcomes". Hoffman and his colleagues asked the AI models to assess the intelligence and employability of people who speak using AAVE compared to people who speak using what they dub "standard American English".

Even when the two sentences had the same meaning, the models were more likely to apply adjectives like "dirty," "lazy," and "stupid" to speakers of AAE than speakers of Standard American English (SAE). The models associated speakers of AAE with less prestigious jobs (or didn't associate them with having a job at all), and when asked to pass judgment on a hypothetical criminal defendant, they were more likely to recommend the death penalty. An even more notable finding may be a flaw the study pinpoints in the ways that researchers try to solve such biases. To purge models of hateful views, companies like OpenAI, Meta, and Google use feedback training, in which human workers manually adjust the way the model responds to certain prompts. This process, often called "alignment," aims to recalibrate the millions of connections in the neural network and get the model to conform better with desired values. The method works well to combat overt stereotypes, and leading companies have employed it for nearly a decade.