W. Scott Stornetta Stanford University, Physics Department, Stanford, Ca., 94305 Tad Hogg and B. A. Huberman Xerox Palo Alto Research Center, Palo Alto, Ca. 94304 ABSTRACT Recognizing patterns with temporal context is important for such tasks as speech recognition, motion detection and signature verification. We propose an architecture in which time serves as its own representation, and temporal context is encoded in the state of the nodes. We contrast this with the approach of replicating portions of the architecture to represent time. As one example of these ideas, we demonstrate an architecture with capacitive inputs serving as temporal feature detectors in an otherwise standard back propagation model. Experiments involving motion detection and word discrimination serve to illustrate novel features of the system.

(June 1987).

TIME-SEQUENTIAL SELF-ORGANIZATION OF HIERARCHICAL NEURAL NETWORKS Ronald H. Silverman Cornell University Medical College, New York, NY 10021 Andrew S. Noetzel polytechnic University, Brooklyn, NY 11201 ABSTRACT Self-organization of multi-layered networks can be realized by time-sequential organization of successive neural layers. Lateral inhibition operating in the surround of firing cells in each layer provides for unsupervised capture of excitation patterns presented by the previous layer. By presenting patterns of increasing complexity, in coordination with network selforganization, higher levels of the hierarchy capture concepts implicit in the pattern set. INTRODUCTION A fundamental difficulty in self-organization of hierarchical, multi-layered, networks of simple neuron-like cells is the determination of the direction of adjustment of synaptic link weights between neural layers not directly connected to input or output patterns. Several different approaches have been used to address this problem.

The net is defined as a set of units, each witb an activation, and weights between units which determine the activations. The algorithm uses a gradient descent technique to calculate the direction by which each weight should be changed in order to minimise the summed squared difference between the desired output and the actual output. Using this algorithm it is believed that a net can be trained to make an arbitrary nonlinear mapping of the input units onto the output units if given enough intermediate units. This'static' net can be used as part of a larger system with more complex behaviour. The static net has no memory for past inputs, but many problems require the context of the input in order to c.ompute the answer.

Box 150, Cheongryang, Seoul, Korea ABSTRACT Inverse matrix calculation can be considered as an optimization. We have demonstrated that this problem can be rapidly solved by highly interconnected simple neuron-like analog processors. A network for matrix inversion based on the concept of Hopfield's neural network was designed, and implemented with electronic hardware. With slight modifications, the network is readily applicable to solving a linear simultaneous equation efficiently. Notable features of this circuit are potential speed due to parallel processing, and robustness against variations of device parameters.

The generalization replaces two state neurons by neurons taking a richer set of values. Two classes of neuron input output relations are developed guaranteeing convergence to stable states. The first is a class of "continuous" relations and the second is a class of allowed quantization rules for the neurons.

ABSTRACT Advances in brain theory need two complementary approaches: Analytical investigations by in situ measurements and as well synthetic modelling supported by computer simulations to generate suggestive hypothesis on purposeful structures in the neural tissue. In this paper research of the second line is described: Starting from a neurophysiologically inspired model of stimulusresponse (SR) and/or associative memorization and a psychologically motivated ministructure for basic control tasks, preconditions and conditions are studied for cooperation of such units in a hierarchical organisation, as can be assumed to be the general layout of macrostructures in the brain. I. INTRODUCTION Theoretic modelling in brain theory is a highly speculative subject. However, it is necessary since it seems very unlikely to get a clear picture of this very complicated device by just analyzing the available measurements on sound and/or damaged brain parts only. As in general physics, one has to realize, that there are different levels of modelling: in physics stretching from the atomary level over atom assemblies till up to general behavioural models like kinematics and mechanics, in brain theory stretching from chemical reactions over electrical spikes and neuronal cell assembly cooperation till general human behaviour.

ABSTRACT Artificial neural networks (ANNs) are capable of accurate recognition of simple speech vocabularies such as isolated digits [1]. This paper looks at two more difficult vocabularies, the alphabetic E-set and a set of polysyllabic words. The E-set is difficult because it contains weak discriminants and polysyllables are difficult because of timing variation. Polysyllabic word recognition is aided by a time pre-alignment technique based on dynamic programming and E-set recognition is improved by focusing attention. Recognition accuracies are better than 98% for both vocabularies when implemented with a single layer perceptron.

Lateral inhibition operating in the surround of firing cells in each layer provides for unsupervised capture of excitation patterns presented by the previous layer. By presenting patterns of increasing complexity, in coordination with network selforganization, higherlevels of the hierarchy capture concepts implicit in the pattern set. INTRODUCTION A fundamental difficulty in self-organization of hierarchical, multi-layered, networks of simple neuron-like cells is the determination of the direction of adjustment of synaptic link weights between neural layers not directly connected to input or output patterns. Several different approaches have been used to address this problem. One is to provide teaching inputs to the cells in internal layers of the hierarchy.

Here, X is a finite or infinite set, and Y is another finite or infinite set. A learning machine observes any set of pairs (x, y) sampled randomly from X x Y. (X x Y means the Cartesian product of X and Y.) And, it computes some estimate j: