Managing the Cybersecurity Vulnerabilities of Artificial Intelligence

Last week, Andy Grotto and I published a new working paper on policy responses to the risk that artificial intelligence (AI) systems, especially those dependent on machine learning (ML), can be vulnerable to intentional attack. As the National Security Commission on Artificial Intelligence found, "While we are on the front edge of this phenomenon, commercial firms and researchers have documented attacks that involve evasion, data poisoning, model replication, and exploiting traditional software flaws to deceive, manipulate, compromise, and render AI systems ineffective." The demonstrations of vulnerability are remarkable: In the speech recognition domain, research has shown it is possible to generate audio that sounds like speech to ML algorithms but not to humans. There are multiple examples of tricking image recognition systems to misidentify objects using perturbations that are imperceptible to humans, including in safety critical contexts (such as road signs). One team of researchers fooled three different deep neural networks by changing just one pixel per image.