The car's underlying AI technology, known as deep learning, has proved very powerful at solving problems in recent years, and it has been widely deployed for tasks like image captioning, voice recognition, and language translation. There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen--or shouldn't happen--unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users. Otherwise it will be hard to predict when failures might occur--and it's inevitable they will. That's one reason Nvidia's car is still experimental.

The car's underlying AI technology, known as deep learning, has proved very powerful at solving problems in recent years, and it has been widely deployed for tasks like image captioning, voice recognition, and language translation. There is now hope that the same techniques will be able to diagnose deadly diseases, make million-dollar trading decisions, and do countless other things to transform whole industries. But this won't happen--or shouldn't happen--unless we find ways of making techniques like deep learning more understandable to their creators and accountable to their users. Otherwise it will be hard to predict when failures might occur--and it's inevitable they will. That's one reason Nvidia's car is still experimental.

Since its founding, the Defense Advanced Research Projects Agency (DARPA) has been a hub of innovation. While created as the research arm of the Department of Defense, DARPA has played an important role in some of the technologies that have become (or will become) fundamental to modern human societies. In the 1960s and 1970s, DARPA (then known as ARPA), created ARPANET, the computer network that became the precursor to the internet. In 2003, DARP launched CALO, a project that ushered in the era of Siri and other voice-enabled assistants. In 2004, DARPA launched the Grand Challenge, a competition that set the stage for current developments and advances in self-driving cars. In 2013, DARPA launched the Brain Initiative, an ambitious project that brings together universities, tech companies and neuroscientists to discover how the brain works and develop technologies that enable the human brain to interact with the digital world. Among DARPA's many exciting projects is Explainable Artificial Intelligence (XAI), an initiative launched in 2016 aimed at solving one of the principal challenges of deep learning and neural networks, the subset of AI that is becoming increasing prominent in many different sectors.

Artificial Intelligence (AI) systems are taking over a vast array of tasks that previously depended on human expertise and judgment. Often, however, the "reasoning" behind their actions is unclear, and can produce surprising errors or reinforce biased processes. One way to address this issue is to make AI "explainable" to humans--for example, designers who can improve it or let users better know when to trust it. Although the best styles of explanation for different purposes are still being studied, they will profoundly shape how future AI is used. Some explainable AI, or XAI, has long been familiar, as part of online recommender systems: book purchasers or movie viewers see suggestions for additional selections described as having certain similar attributes, or being chosen by similar users.

Today's machine learning systems are more advanced than ever, capable of automating increasingly complex tasks and serving as a critical tool for human operators. Despite recent advances, however, a critical component of Artificial Intelligence (AI) remains just out of reach – machine common sense. Defined as "the basic ability to perceive, understand, and judge things that are shared by nearly all people and can be reasonably expected of nearly all people without need for debate," common sense forms a critical foundation for how humans interact with the world around them. Possessing this essential background knowledge could significantly advance the symbiotic partnership between humans and machines. But articulating and encoding this obscure-but-pervasive capability is no easy feat.

Machine learning systems are more advanced than they ever have been, but a critical component is still missing: machine common sense. Machine common sense is "the basic ability to perceive, understand, and judge things that are shared by nearly all people and can be reasonably expected of nearly all people without need for debate." DARPA believes that possessing this knowledge could significantly advance the relationship between humans and machines, but encoding this capability is a difficult task. "The absence of common sense prevents an intelligent system from understanding its world, communicating naturally with people, behaving reasonably in unforeseen situations, and learning from new experiences," said Dave Gunning, a program manager in DARPA's Information Innovation Office (I2O). "This absence is perhaps the most significant barrier between the narrowly focused AI applications we have today and the more general AI applications we would like to create in the future."

Two-and-a-half years ago, technology wizard and Stanford University Master of Science in Computer Science graduate David Gunningjoined with DARPA, the Defense Advanced Research Projects Agency, to manage a program to develop explainable artificial intelligence. And listen up: The XAI arena, as it's abbreviated, is where we want to head -- this is where technology development ought to focus. XAI is the common-sense older brother in a digitized world filled with flashy, privacy-invading, data-gobbling gadgets and machine-controlling bullies. The goal of XAI, Gunning said, in a recent telephone conversation, is not so much to "take human thinking and put it into machines," as nearly all of today's artificial intelligence seeks to do. Rather, XAI's aim is to equip the machine with the ability to tell its human operators why it arrives at the conclusions it does -- to make the machine explain itself, so to speak. That means humans still stay at the helm.

Intelligence agents and military operatives may come to rely heavily on Machine Learning and Artificial Intelligence (AI) to parse huge quantities of data, and to control a growing arsenal of autonomous systems, but the US Military wants to make sure that this doesn't lead to blindly trusting algorithms, that even though there are a couple of tests to assess how dangerous they are, or could become, are still at their heart mysterious black boxes. As a result the Defense Advanced Research Projects Agency (DARPA), a division of the US Defense Department that explores new technologies, is following the lead shown by Columbia University, MIT, and Nvidia, who have all been trying to develop new systems that read AI's minds and get them to explain their decision making processes, and they've announced they're going to be funding several new projects. The approaches range from adding further machine learning systems geared toward providing an explanation, to the development of new machine learning approaches that incorporate an "elucidation by design." "We now have this real explosion of AI," says David Gunning, the DARPA program manager who is funding an effort to develop AI techniques that include some explanation of their reasoning, "the reason for that is mainly machine learning, and deep learning in particular." Deep learning and other machine learning techniques have taken Silicon Valley by storm, improving voice recognition and image classification significantly, and they are being used in more contexts than ever before, including areas like law enforcement and medicine, where the consequences of a mistake may be serious.

As Mark Hamill humorously shared the behind-the-scenes of "Star Wars: The Last Jedi" with a packed SXSW audience, two floors below on the exhibit floor Universal Robots recreated General Grievous' famed light saber battles. The battling machines were steps away from a twelve foot dancing Kuka robot and an automated coffee dispensary. Somehow the famed interactive festival known for its late night drinking, dancing and concerts had a very mechanical feel this year. Everywhere debates ensued between utopian tech visionaries and dystopia-fearing humanists. Even my panel on "Investing In The Autonomy Economy" took a very social turn when discussing the opportunities of utilizing robots for the growing aging population.

Critics, however, see it as an effort to blunt outside regulation by cities, states or the federal government, and they question if tech companies are best suited to shape the rules of the road. For the corporations, the algorithms will be proprietary tools to assess your loan-worthiness, your job application, and your risk of stroke. Many balk at the costs of developing systems that not only learn to make decisions, but that also explain those decisions to outsiders. When New York City proposed a law in August requiring that companies publish source code for algorithms used by city agencies, tech firms pushed back, saying they needed to protect proprietary algorithms. The city passed a scaled-back version in December without the source-code requirement.