How AI Can Accidentally Recreate Prejudices From the Real World

Last year, it was revealed that Amazon's facial recognition software, 'Rekognition' matched Congresspeople's headshots with photos from inside a mugshot database. In total, 28 members of Congress were falsely identified as other people that had previously been arrested for committing criminal offenses. The false matches were disproportionate toward people of color, including six members of the Congressional Black Caucus, among these was civil rights activist Rep. John Lewis. In July of 2018, the American Civil Liberties Union conducted an independent analysis using the same default settings that Amazon's Rekognition software uses. This analysis ran a check on the whole of Congress against 25,000 publicly available arrest photographs. The results showed that 40% of the false matches revealed in this analysis were people of color, even though they account for only 20% of Congress.