Weighing the Trade-Offs of Explainable AI

In 1997, IBM supercomputer Deep Blue made a move against chess champion Garry Kasparov that left him stunned. The computer's choice to sacrifice one of its pieces seemed so inexplicable to Kasparov that he assumed it was a sign of the machine's superior intelligence. Shaken, he went on to resign his series against the computer, even though he had the upper hand. Fifteen years later, however, one of Deep Blue's designers revealed that fateful move wasn't the sign of advanced machine intelligence -- it was the result of a bug. Today, no human can beat a computer at chess, but the story still underscores just how easy it is to blindly trust AI when you don't know what's going on.