Allen School News » Allen School researchers find racial bias built into hate-speech detection

The volume of content posted on Facebook, YouTube, Twitter and other social media platforms every moment of the day, from all over the world, is monumental. Unfortunately, some of it is biased, hate-filled language targeting members of minority groups and often prompting violent action against them. Because it is impossible for human moderators to keep up with the volume of content generated in real-time, platforms are turning to artificial intelligence and machine learning to catch toxic language and stop it quickly. Regrettably, these toxic language finding tools have been found to suppress already marginalized voices. "Despite the benevolent intentions of most of these efforts, there's actually a really big racial bias problem in hate speech detection right now," said Maarten Sap, a Ph.D. student in the Allen School.