DARPA funds programs to get black box AI's to explain their decisions

Intelligence agents and military operatives may come to rely heavily on Machine Learning and Artificial Intelligence (AI) to parse huge quantities of data, and to control a growing arsenal of autonomous systems, but the US Military wants to make sure that this doesn't lead to blindly trusting algorithms, that even though there are a couple of tests to assess how dangerous they are, or could become, are still at their heart mysterious black boxes. As a result the Defense Advanced Research Projects Agency (DARPA), a division of the US Defense Department that explores new technologies, is following the lead shown by Columbia University, MIT, and Nvidia, who have all been trying to develop new systems that read AI's minds and get them to explain their decision making processes, and they've announced they're going to be funding several new projects. The approaches range from adding further machine learning systems geared toward providing an explanation, to the development of new machine learning approaches that incorporate an "elucidation by design." "We now have this real explosion of AI," says David Gunning, the DARPA program manager who is funding an effort to develop AI techniques that include some explanation of their reasoning, "the reason for that is mainly machine learning, and deep learning in particular." Deep learning and other machine learning techniques have taken Silicon Valley by storm, improving voice recognition and image classification significantly, and they are being used in more contexts than ever before, including areas like law enforcement and medicine, where the consequences of a mistake may be serious.