Operational AI systems (for example, self-driving cars) need to obey both the law of the land and our values. We propose AI oversight systems ("AI Guardians") as an approach to addressing this challenge, and to respond to the potential risks associated with increasingly autonomous AI systems.a These AI oversight systems serve to verify that operational systems did not stray unduly from the guidelines of their programmers and to bring them back in compliance if they do stray. The introduction of such second-order, oversight systems is not meant to suggest strict, powerful, or rigid (from here on'strong') controls. Operations systems need a great degree of latitude in order to follow the lessons of their learning from additional data mining and experience and to be able to render at least semi-autonomous decisions (more about this later).
Stephen Hawking has warned that technology needs to be controlled in order to prevent it from destroying the human race. The world-renowned physicist, who has spoken out about the dangers of artificial intelligence in the past, believes we need to establish a way of identifying threats quickly, before they have a chance to escalate. "Since civilisation began, aggression has been useful inasmuch as it has definite survival advantages," he told The Times. "It is hard-wired into our genes by Darwinian evolution. Now, however, technology has advanced at such a pace that this aggression may destroy us all by nuclear or biological war.
One response to the call by experts in robotics and artificial intelligence for an ban on "killer robots" ("lethal autonomous weapons systems" or Laws in the language of international treaties) is to say: shouldn't you have thought about that sooner? Figures such as Tesla's CEO, Elon Musk, are among the 116 specialists calling for the ban. "We do not have long to act," they say. "Once this Pandora's box is opened, it will be hard to close." But such systems are arguably already here, such as the "unmanned combat air vehicle" Taranis developed by BAE and others, or the autonomous SGR-A1 sentry gun made by Samsung and deployed along the South Korean border.
The emerging technologies of the Fourth Industrial Revolution (4IR) will inevitably transform the world in many ways – some that are desirable and others that are not. The extent to which the benefits are maximized and the risks mitigated will depend on the quality of governance – the rules, norms, standards, incentives, institutions, and other mechanisms that shape the development and deployment of each particular technology. Too often the debate about emerging technologies takes place at the extremes of possible responses: among those who focus intently on the potential gains and others who dwell on the potential dangers. The real challenge lies in navigating between these two poles: building understanding and awareness of the trade-offs and tensions we face, and making informed decisions about how to proceed. This task is becoming more pressing as technological change deepens and accelerates, and as we become more aware of the lagged societal, political and even geopolitical impact of earlier waves of innovation.
This is a story about the future. Not the unhappy future, the one where climate change turns the planet into a cinder or we all die in a global nuclear war. This is the happy version. It's the one where computers keep getting smarter and smarter, and clever engineers keep building better and better robots. Plus they're computers: They never get tired, they're never ill-tempered, they never make mistakes, and they have instant access to all of human knowledge. Global warming is a problem of the past because computers have figured out how to generate limitless amounts of green energy and intelligent robots have tirelessly built the infrastructure to deliver it to our homes. No one needs to work anymore. Robots can do everything humans can do, and they do it uncomplainingly, 24 hours a day. Some things remain scarce--beachfront property in Malibu, original Rembrandts--but thanks to super-efficient use of natural resources and massive recycling, scarcity of ordinary consumer goods is a thing of the past. Our days are spent however we please, perhaps in study, perhaps playing video games. Maybe you think I'm pulling your leg here.