Understanding and Avoiding AI Failures: A Practical Guide

With current AI technologies, harm done by AIs is limited to power that we put directly in their control. As said in [59], "For Narrow AIs, safety failures are at the same level of importance as in general cybersecurity, but for AGI it is fundamentally different." Despite AGI (artificial general intelligence) still being well out of reach, the nature of AI catastrophes has already changed in the past two decades. Automated systems are now not only malfunctioning in isolation, they are interacting with humans and with each other in real time. This shift has made traditional systems analysis more difficult, as AI has more complexity and autonomy than software has before. In response to this, we analyze how risks associated with complex control systems have been managed historically and the patterns in contemporary AI failures to what kinds of risks are created from the operation of any AI system. We present a framework for analyzing AI systems before they fail to understand how they change the risk landscape of the systems they are embedded in, based on conventional system analysis and open systems theory as well as AI safety principles. Finally, we present suggested measures that should be taken based on an AI system's properties. Several case studies from different domains are given as examples of how to use the framework and interpret its results.