The net is defined as a set of units, each witb an activation, and weights between units which determine the activations. The algorithm uses a gradient descent technique to calculate the direction by which each weight should be changed in order to minimise the summed squared difference between the desired output and the actual output. Using this algorithm it is believed that a net can be trained to make an arbitrary nonlinear mapping of the input units onto the output units if given enough intermediate units. This'static' net can be used as part of a larger system with more complex behaviour. The static net has no memory for past inputs, but many problems require the context of the input in order to c.ompute the answer.

REFLEXIVE ASSOCIATIVE MEMORIES Hendrlcus G. Loos Laguna Research Laboratory, Fallbrook, CA 92028-9765 ABSTRACT In the synchronous discrete model, the average memory capacity of bidirectional associative memories (BAMs) is compared with that of Hopfield memories, by means of a calculat10n of the percentage of good recall for 100 random BAMs of dimension 64x64, for different numbers of stored vectors. The memory capac1ty Is found to be much smal1er than the Kosko upper bound, which Is the lesser of the two dimensions of the BAM. On the average, a 64x64 BAM has about 68 % of the capacity of the corresponding Hopfield memory with the same number of neurons. The memory capacity limitations are due to spurious stable states, which arise In BAMs In much the same way as in Hopfleld memories. Occurrence of spurious stable states can be avoided by replacing the thresholding in the backlayer of the BAM by another nonl1near process, here called "Dominant Label Selection" (DLS).

As a means of probing these cerebellar mechanisms, my colleagues and I have been conducting microelectrode studies of the neural messages that flow through the intermediate division of the cerebellum and onward to limb muscles via the rubrospinal tract. We regard this cerebellorubrospinal pathway as a useful model system for studying general problems of sensorimotor integration and adaptive brain function.

Current Focus Of Learning Research Most connectionist learning algorithms may be grouped into three general catagories, commonly referred to as supenJised, unsupenJised, and reinforcement learning. Supervised learning requires the explicit participation of an intelligent teacher, usually to provide the learning system with task-relevant input-output pairs (for two recent examples, see [1,2]). Unsupervised learning, exemplified by "clustering" algorithms, are generally concerned with detecting structure in a stream of input patterns [3,4,5,6,7]. In its final state, an unsupervised learning system will typically represent the discovered structure as a set of categories representing regions of the input space, or, more generally, as a mapping from the input space into a space of lower dimension that is somehow better suited to the task at hand. In reinforcement learning, a "critic" rewards or penalizes the learning system, until the system ultimately produces the correct output in response to a given input pattern [8]. It has seemed an inevitable tradeoff that systems needing to rapidly learn specific, behaviorally useful input-output mappings must necessarily do so under the auspices of an intelligent teacher with a ready supply of task-relevant training examples. This state of affairs has seemed somewhat paradoxical, since the processes of Rerceptual and cognitive development in human infants, for example, do not depend on the moment by moment intervention of a teacher of any sort. Learning by Doing The current work has been focused on a fourth type of learning algorithm, i.e. learning-bydoing, an approach that has been very little studied from either a connectionist perspective

The high interconnectivity required by neural computers can be simply implemented in optics because channels for optical signals may be superimposed in three dimensions with little or no cross coupling. Since these channels may be formed holographically, optical neural systems can be designed to create and maintain interconnections very simply. Thus the optical system designer can to a large extent avoid the analytical and topological problems of determining individual interconnections for a given neural system and constructing physical paths for these interconnections. An archetypical design for a single layer of an optical neural computer is shown in Figure 1. Nonlinear thresholding elements, neurons, are arranged on two dimensional planes which are interconnected via the third dimension by holographic elements. The key concerns in implementing this design involve the need for suitable nonlinearities for the neural planes and high capacity, easily modifiable holographic elements. While it is possible to implement the neural function using entirely optical nonlinearities, for example using etalon arrays\ optoelectronic two dimensional spatial light modulators (2D SLMs) suitable for this purpose are more readily available.

Many optimization models of neural networks need constraints to restrict the space of outputs to a subspace which satisfies external criteria. Optimizations using energy methods yield "forces" which act upon the state of the neural network. The penalty method, in which quadratic energy constraints are added to an existing optimization energy, has become popular recently, but is not guaranteed to satisfy the constraint conditions when there are other forces on the neural model or when there are multiple constraints. In this paper, we present the basic differential multiplier method (BDMM), which satisfies constraints exactly; we create forces which gradually apply the constraints over time, using "neurons" that estimate Lagrange multipliers. The basic differential multiplier method is a differential version of the method of multipliers from Numerical Analysis.

TIME-SEQUENTIAL SELF-ORGANIZATION OF HIERARCHICAL NEURAL NETWORKS Ronald H. Silverman Cornell University Medical College, New York, NY 10021 Andrew S. Noetzel polytechnic University, Brooklyn, NY 11201 ABSTRACT Self-organization of multi-layered networks can be realized by time-sequential organization of successive neural layers. Lateral inhibition operating in the surround of firing cells in each layer provides for unsupervised capture of excitation patterns presented by the previous layer. By presenting patterns of increasing complexity, in coordination with network selforganization, higher levels of the hierarchy capture concepts implicit in the pattern set. INTRODUCTION A fundamental difficulty in self-organization of hierarchical, multi-layered, networks of simple neuron-like cells is the determination of the direction of adjustment of synaptic link weights between neural layers not directly connected to input or output patterns. Several different approaches have been used to address this problem.

A"'ir Dembo Information Systems Laboratory, Stanford University Stanford, CA 94305 Ofer Zeitouni Laboratory for Information and Decision Systems MIT, Cambridge, MA 02139 ABSTRACT A class of high dens ity assoc iat ive memories is constructed, starting from a description of desired properties those should exhib it. These propert ies include high capac ity, controllable bas ins of attraction and fast speed of convergence. Fortunately enough, the resulting memory is implementable by an artificial Neural Net. I NfRODUCTION Most of the work on assoc iat ive memories has been structure oriented, i.e.. given a Neural architecture, efforts were directed towards the analysis of the resulting network. Issues like capacity, basins of attractions, etc. were the main objects to be analyzed cf., e.g.

The network model considered consists of interconnected groups of neurons, where each group could be fully interconnected (it could have feedback connections, with possibly asymmetric weights), but no loops between the groups are allowed. A stochastic descent algorithm is applied, under a certain inequality constraint on each intragroup weight matrix which ensures for the network to possess a unique equilibrium state for every input. Introduction It has been shown in the last few years that large networks of interconnected "neuron" -like elemp.nts

(June 1987).