Given a set of input-output training samples, we describe a procedure for determining the time sequence of weights for a dynamic neural network to model an arbitrary input-output process. We formulate the input-output mapping problem as an optimal control problem, defining a performance index to be minimized as a function of time-varying weights. We solve the resulting nonlinear two-point-boundary-value problem, and this yields the training rule. For the performance index chosen, this rule turns out to be a continuous time generalization of the outer product rule earlier suggested heuristically by Hopfield for designing associative memories. Learning curves for the new technique are presented.

It has been known for many years that specific regions of the working cerebral cortex display periodic variations in correlated cellular activity. While the olfactory system has been the focus of much of this work, similar behavior has recently been observed in primary visual cortex. We have developed models of both the olfactory and visual cortex which replicate the observed oscillatory properties of these networks. Using these models we have examined the dependence of oscillatory behavior on single cell properties and network architectures. We discuss the idea that the oscillatory events recorded from cerebral cortex may be intrinsic to the architecture of cerebral cortex as a whole, and that these rhythmic patterns may be important in coordinating neuronal activity during sensory processmg.

To achieve high-rate image data compression while maintainig a high quality reconstructed image, a good image model and an efficient way to represent the specific data of each image must be introduced. Based on the physiological knowledge of multi - channel characteristics and inhibitory interactions between them in the human visual system, a mathematically coherent parallel architecture for image data compression which utilizes the Markov random field Image model and interactions between a vast number of filter banks, is proposed.

Analytic solutions to the information-theoretic evolution equation of the connection strength of a three-layer feedforward neural net for visual information processing are presented. The results are (1) the receptive fields of the feature-analysing cells correspond to the eigenvector of the maximum eigenvalue of the Fredholm integral equation of the first kind derived from the evolution equation of the connection strength; (2) a symmetry-breaking mechanism (parity-violation) has been identified to be responsible for the changes of the morphology of the receptive field; (3) the conditions for the formation of different morphologies are explicitly identified.

The effects of parameter modifications imposed by hardware constraints on a self-organizing feature map algorithm were examined. Performance was measured by the error rate of a speech recognition system which included this algorithm as part of the front-end processing. System parameters which were varied included weight (connection strength) quantization, adap tation quantization, distance measures and circuit approximations which include device characteristics and process variability. Experiments using the TI isolated word database for 16 speakers demonstrated degradation in performance when weight quantization fell below 8 bits. The competitive nature of the algorithm rela..xes constraints on uniformity and linearity which makes it an excellent candidate for a fully analog circuit implementation. Prototype circuits have been fabricated and characterized following the constraints established through the simulation efforts. 1 Introduction The self-organizing feature map algorithm developed by Kohonen [Kohonen, 1988] readily lends itself to the task of vector quantization for use in such areas as speech recognition.

This paper explores the use of a model neural network for motor learning. Steinbuch and Taylor presented neural network designs to do nearest neighbor lookup in the early 1960s. In this paper their nearest neighbor network is augmented with a local model network, which fits a local model to a set of nearest neighbors. The network design is equivalent to local regression. This network architecture can represent smooth nonlinear functions, yet has simple training rules with a single global optimum.

We present a number of Time-Delay Neural Network (TDNN) based architectures for multi-speaker phoneme recognition (/b,d,g/ task). We use speech of two females and four males to compare the performance of the various architectures against a baseline recognition rate of 95.9% for a single IDNN on the six-speaker /b,d,g/ task. This series of modular designs leads to a highly modular multi-network architecture capable of performing the six-speaker recognition task at the speaker dependent rate of 98.4%. In addition to its high recognition rate, the so-called "Meta-Pi" architecture learns - without direct supervision - to recognize the speech of one particular male speaker using internal models of other male speakers exclusively.

We study networks of spiking neurons in which spikes are fired as a Poisson process. The state of a cell is determined by the instantaneous firing rate, and in the limit of high firing rates our model reduces to that studied by Hopfield. We find that the inclusion of spiking results in several new features, such as a noise-induced asymmetry between "on" and "off" states of the cells and probability currents which destroy the usual description of network dynamics in terms of energy surfaces. Taking account of spikes also allows us to calibrate network parameters such as "synaptic weights" against experiments on real synapses. Realistic forms of the post synaptic response alters the network dynamics, which suggests a novel dynamical learning mechanism.

References Baum, E. and Haussler, D. (1989) What size net gives valid generalization? in D. S. Touretzky (Ed.) Advances in neural information processing systems I, Morgan Kaufman.

Analog neurons with limited precision essentially compute k-ary weighted multilinear threshold functions.