Backprop Diffusion is Biologically Plausible

The Backpropagation algorithm relies on the abstraction of using a neural model that gets rid of the notion of time, since the input is mapped instantaneously to the output. In this paper, we cl aim that this abstraction of ignoring time, along with the abrupt inp ut changes that occur when feeding the training set, are in fact the reas ons why, in some papers, Backprop biological plausibility is regarded as an arguable issue. We show that as soon as a deep feedforward network oper ates with neurons with time-delayed response, the backprop weig ht update turns out to be the basic equation of a biologically plausibl e diffusion process based on forward-backward waves. We also show that s uch a process very well approximates the gradient for inputs that are not too fast with respect to the depth of the network. These remarks s omewhat disclose the diffusion process behind the backprop equation and leads us to interpret the corresponding algorithm as a degenerati on of a more general diffusion process that takes place also in neural net works with cyclic connections.