Amazon's facial recognition technology falsely identified 28 members of Congress as people who have been arrested for crimes, according to the American Civil Liberties Union (ACLU). The ACLU of Northern California's test of Amazon's controversial Rekognition software also found that people of color were disproportionately misidentified in a mugshot database, raising new concerns about racial bias and the potential for abuse by law enforcement. The report followed revelations in May that Amazon has been marketing and selling the Rekognition technology to police agencies, leading privacy advocates to urge CEO Jeff Bezos to stop providing the product to the government. "Our test reinforces that face surveillance is not safe for government use," Jacob Snow, a technology and civil liberties attorney at the ACLU Foundation of Northern California, said in a statement. "Face surveillance will be used to power discriminatory surveillance and policing that targets communities of color, immigrants, and activists.
Amazon's face surveillance technology is the target of growing opposition nationwide, and today, there are 28 more causes for concern. In a test the ACLU recently conducted of the facial recognition tool, called "Rekognition," the software incorrectly matched 28 members of Congress, identifying them as other people who have been arrested for a crime. The members of Congress who were falsely matched with the mugshot database we used in the test include Republicans and Democrats, men and women, and legislators of all ages, from all across the country. Our test used AmazonRekognition to compare images of members of Congress with a database of mugshots. The results included 28 incorrect matches.
Future Tense is a partnership of Slate, New America, and Arizona State University that examines emerging technologies, public policy, and society. The ACLU released a report on Thursday revealing that Rekognition, Amazon's facial recognition tool, had falsely matched 28 members of Congress to mug shots. Members of the ACLU purchased the version of Rekognition that Amazon offers to the general public and ran public photos of every member of the House and Senate against a database of 25,000 arrest photos. The entire experiment costed $12.33, which, as ACLU attorney Jake Snow writes in a blogpost, is "less than a large pizza." Almost 40 percent of the representatives that Rekognition falsely matched were people of color, even though they make up only 20 percent of Congress.
Amazon defended the law enforcement use cases for its facial recognition technology Friday in an effort to throw water on fears raised by civil rights activists. Matt Wood, a leader on the Amazon Web Services machine learning team, published a blog post in response to criticism from the American Civil Liberties Union and other advocacy groups, which have been demanding the company stop selling its Rekognition software to police. In the post, Wood cautions that we "should not throw away the oven because the temperature could be set wrong and burn the pizza." Wood expressed skepticism about an experiment the ACLU conducted using Rekognition to compare headshots of the members of Congress with a database of 25,000 mugshots. The test set Rekognition to make matches with an 80 percent confidence rating, according to Amazon.
Amazon touts its Rekognition facial recognition system as "simple and easy to use," encouraging customers to "detect, analyze, and compare faces for a wide variety of user verification, people counting, and public safety use cases." And yet, in a study released Thursday by the American Civil Liberties Union, the technology managed to confuse photos of 28 members of Congress with publicly available mug shots. Given that Amazon actively markets Rekognition to law enforcement agencies across the US, that's simply not good enough. The ACLU study also illustrated the racial bias that plagues facial recognition today. "Nearly 40 percent of Rekognition's false matches in our test were of people of color, even though they make up only 20 percent of Congress," wrote ACLU attorney Jacob Snow.