Review for NeurIPS paper: Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

The contributions are well summarized by R2's comments, "The work is motivated by arguing that DFA was so far only used on small datasets, and was shown to not perform well on computer vision tasks, in part because of the usage of CNNs in these settings. This survey challenges these views by conducting an extensive set of experiments using DFA to train s.o.t.a. The benchmarks include view synthesis, language modeling, recommender systems, and graph embedding. They compare the performance of these models to ones trained using a normal BP approach. The authors show that DFA can be competitive to classical BP in many scenarios, and also show how further improvements could be implemented.