The Past and Future of Deep Learning

It turns out that the Boltzmann machines of that era did not scale up well, but other architectures designed by Hinton, LeCun, Bengio, Olshausen, Osindero, Sutskever, Courville, Ng, and others did. Was it the one-layer-at-a-time training technique? GPU clusters that allow faster training? I can't say for sure, and I hope that continued analysis will give us a better picture. But I can say that in speech recognition, computer vision object recognition, the game of Go, and other fields, the difference has been dramatic: error rates go down when you use deep learning, and both these fields have undergone a complete transformation in the last few years: essentially all the teams have chosen deep learning, because it just works.