Global Big Data Conference

In 2017, Google's Counter Abuse Technology team and Jigsaw, the organization working under Google parent company Alphabet to tackle cyberbullying and disinformation, released an AI-powered API for content moderation called Perspective. Perspective's goal is to "identify toxic comments that can undermine a civil exchange of ideas," offering a score from zero to 100 on how similar new comments are to others previously identified as toxic, defined as how likely a comment is to make someone leave a conversation. Jigsaw claims its AI can immediately generate an assessment of a phrase's toxicity more accurately than any keyword blacklist and faster than any human moderator. But studies show that technologies similar to Jigsaw's still struggle to overcome major challenges, including biases against specific subsets of users. For example, a team at Penn State recently found that posts on social media about people with disabilities could be flagged as more negative or toxic by commonly used, public sentiment, and toxicity detection models.