A Forward DNI

In this paper we have focused on the backward (or feedback) DNI, but there is another interesting paradigm between two neural networks dubbed "forward" DNI. Here we describe this variant of the model and below its link to the cerebellum. The difference to backward DNI is that now the synthesiser predicts forward activity, not backward. Though more nuanced, the goal of forward DNI as presented in [5] is also to hasten learning. As an example, suppose we have a feedforward network as the main model and equip a backward synthesiser (one which predicts same layer error gradients) at each layer as well as a forward synthesiser which projects from the original network input x onto each layer.