New MIT technique reveals the basis for machine-learning systems' hidden decisions

A Stanford School of Medicine machine-learning method for automatically analyzing images of cancerous tissues and predicting patient survival was found more accurate than doctors in breast-cancer diagnosis, but doctors still don't trust this method, say MIT researchers (credit: Science/AAAS) MIT researchers have developed a method to determine the rationale for predictions by neural networks, which loosely mimic the human brain. Neural networks, such as Google's Alpha Go program, use a process known as "deep learning" to look for patterns in training data. An ongoing problem with neural networks is that they are "black boxes." After training, a network may be very good at classifying data, but even its creators will have no idea why. With visual data, it's sometimes possible to automate experiments that determine which visual features a neural net is responding to, but text-processing systems tend to be more opaque.