A methodology for faster supervised learning in dynamical nonlin(cid:173) ear neural networks is presented. It exploits the concept of adjoint operntors to enable computation of changes in the network's re(cid:173) sponse due to perturbations in all system parameters, using the so(cid:173) lution of a single set of appropriately constructed linear equations. The lower bound on speedup per learning iteration over conven(cid:173) tional methods for calculating the neuromorphic energy gradient is O(N2), where N is the number of neurons in the network.

A dynamical neural network consists of a set of interconnected neurons that interact over time continuously. It can exhibit computational properties in the sense that the dynamical system's evolution and/or limit points in the associated state space can correspond to numerical solutions to certain mathematical optimization or learning problems. Such a computational system is particularly attractive in that it can be mapped to a massively parallel computer architecture for power and throughput efficiency, especially if each neuron can rely solely on local information (i.e., local memory). Deriving gradients from the dynamical network's various states while conforming to this last constraint, however, is challenging. We show that by combining ideas of top-down feedback and contrastive learning, a dynamical network for solving the l1-minimizing dictionary learning problem can be constructed, and the true gradients for learning are provably computable by individual neurons. Using spiking neurons to construct our dynamical network, we present a learning process, its rigorous mathematical analysis, and numerical results on several dictionary learning problems.