r/MachineLearning - [R] Learning without feedback: Direct random target projection as a feedback-alignment algorithm with layerwise feedforward training

As there have been some interesting discussions on the alternatives to backpropagation lately (e.g. this reddit thread), I am sharing our latest work just made available on arXiv: Summary: Building on feedback-alignment algorithms, we show how to train multi-layer neural networks using random projections of the target vector, which enables layerwise weight updates using only local and feedforward information. The proposed algorithm is called direct random target projection (DRTP). While backpropagation (BP) requires forward and backward weight symmetry (i.e. Indeed, estimating the layerwise loss gradients only requires a label-dependent random vector selection, making adaptive smart sensors and edge computing the ideal applications due to limited power and computing resources. Despite its simplicity, we demonstrate on the MNIST and CIFAR-10 datasets that DRTP performs close to BP, feedback alignment (FA), direct feedback alignment (DFA) algorithms.