Goto

Collaborating Authors

 Energy






Appendix We first provide additional elements to corroborate our findings: alignment measurement (Section

Neural Information Processing Systems

We report values measured at the deepest DFA layer. Table A.1: Alignment cosine similarity (higher is better, standard deviation in parenthesis) of Table A.2: Alignment cosine similarity (standard deviation in parenthesis) of various graph convolutions architectures as measured on the Cora dataset. We compare DFA to BP, but also to shallow learning-where only the topmost layer is trained. On a simple task like MNIST, a shallow baseline may be as high as 90%. Furthermore, the network is cut down to 3 layers of half the width of NeRF, and no coarse network is used to inform the sampling.


Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Neural Information Processing Systems

Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea. It enforces sequential layer updates, thus preventing efficient paral-lelization of the training process.



Overleaf Example

Neural Information Processing Systems

Different from existing works that represent deformation fields by training a general-purpose neural network, we advocate for an approximation based on mesh-free methods. By letting the network learn deformation parameters at a sparse set of positions in space (nodes), we reconstruct the continuous deformation field in a closed-form with guaranteed smoothness.