Reviews: Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input

Neural Information Processing Systems 

The manuscript describes a discrete time reduction of equilibrium prop (EP) which enables the authors to compare the algorithms gradient estimates and performance directly to BPTT. Moreover, the associated reduction in compute cost also enables them to train the first CNN (that I know of) using EP. While EP approximates BP in feedforward networks, it uses neuronal activity of an RNN in equilibrium to propagate target information or error feedback to perform credit assignment. While this work may be less interesting for DL practitioners because it is still more costly than backprop (BP), it is one of the contenders for bio-plausible backprop which is discussed in the literature. In that regard the present work contributes to this discussion meaningfully.