Review for NeurIPS paper: Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Weaknesses: One of the claims of the paper is that DFA can help reduce training time as well as power consumption if implemented correctly. This claim is made in the abstract, introduction, conclusion, as well as in the broader impact section. Since this claim is made in many places of the paper and used as a central argument for studying DFA, it would be helpful to have a more detailed explanation, with quantitative arguments if possible, of what would be the implications of using DFA rather than backpropagation, and what would the challenges to be overcome. With an appropriate implementation on GPUs, what are the expected gains? Denote N the number of processing stages (say N layers if we consider a standard multi layer neural net).