Does Deep Learning Still Need Backpropagation?

When training deep neural networks, the goal is to automatically discover good "internal representations." One of the most widely accepted methods for this is backpropagation, which uses a gradient descent approach to adjust the neural network's weights. Now, researchers from the Victoria University of Wellington School of Engineering and Computer Science have introduced the HSIC (Hilbert-Schemidt independence criterion) bottleneck as an alternative to backpropagation for finding good representations. The new method has several distinct advantages. Instead of solving problems by using the chain rule as traditional backpropagation does, HSIC solves problems layer-by-layer, eliminating problematic vanishing and exploding gradient issues found in backpropagation.