Deep Learning: What Could Go Wrong?

In a field of research where algorithms can misinterpret stop signs as speed limit signs with the addition of minimal graffiti [3], many commentators are wondering whether current artificial intelligence (AI) solutions are sufficiently robust, resilient, and trustworthy. How can the research community quantify and address such issues? Many empirical approaches investigate the generation of adversarial attacks: small, deliberate perturbations to an input that cause dramatic changes in a system's output. Changes that are essentially imperceptible to the human eye may alter predictions in the field of image classification, which has implications in many high-stakes and safety-critical settings. The rise of algorithms that construct attacks--and heuristic techniques that identify or guard against them--has led to a version of conflict escalation wherein attack and defense strategies become increasingly ingenious [10].