Adjoint-Functions and Temporal Learning Algorithms in Neural Networks

The development of learning algorithms is generally based upon the min(cid:173) imization of an energy function. It is a fundamental requirement to com(cid:173) pute the gradient of this energy function with respect to the various pa(cid:173) rameters of the neural architecture, e.g., synaptic weights, neural gain,etc. In principle, this requires solving a system of nonlinear equations for each parameter of the model, which is computationally very expensive. A new methodology for neural learning of time-dependent nonlinear mappings is presented. It exploits the concept of adjoint operators to enable a fast global computation of the network's response to perturbations in all the systems parameters.