LEARNING IN NETWORKS OF NONDETERMINISTIC ADAPTIVE LOGIC ELEMENTS Richard C. Windecker* AT&T Bell Laboratories, Middletown, NJ 07748 ABSTRACT This paper presents a model of nondeterministic adaptive automata that are constructed from simpler nondeterministic adaptive information processing elements. The first half of the paper describes the model. Chief among these properties is that network aggregates of the model elements can adapt appropriately when a single reinforcement channel provides the same positive or negative reinforcement signal to all adaptive elements of the network at the same time. This holds for multiple-input, multiple-output, multiple-layered, combinational and sequential networks. It also holds when some network elements are "hidden" in that their outputs are not directly seen by the external environment. INTRODUCTION There are two primary motivations for studying models of adaptive automata constructed from simple parts. First, they let us learn things about real biological systems whose properties are difficult to study directly: We form a hypothesis about such systems, embody it in a model, and then see if the model has reasonable learning and behavioral properties. In the present work, the hypothesis being tested is: that much of an animal's behavior as determined by its nervous system is intrinsically nondeterministic; that learning consists of incremental changes in the probabilities governing the animal's behavior; and that this is a consequence of the animal's nervous system consisting of an aggregate of information processing elements some of which are individually nondeterministic and adaptive. The second motivation for studying models of this type is to find ways of building machines that can learn to do (artificially) intelligent and practical things.

Centric Models of the Orientation Map in Primary Visual Cortex William Baxter Department of Computer Science, S.U.N.Y. at Buffalo, NY 14620 Bruce Dow Department of Physiology, S.U.N.Y. at Buffalo, NY 14620 Abstract In the visual cortex of the monkey the horizontal organization of the preferred orientations of orientation-selective cells follows two opposing rules: 1) neighbors tend to have similar orientation preferences, and 2) many different orientations are observed in a local region. Several orientation models which satisfy these constraints are found to differ in the spacing and the topological index of their singularities. Using the rate of orientation change as a measure, the models are compared to published experimental results. Introduction It has been known for some years that there exist orientation-sensitive neurons in the visual cortex of cats and mOnkeysl,2. These cells react to highly specific patterns of light occurring in narrowly circumscribed regiOns of the visual field, i.e., the cell's receptive field. The best patterns for such cells are typically not diffuse levels of illumination, but elongated bars or edges oriented at specific angles.

ENCODING GEOMETRIC INVARIANCES IN HIGHER-ORDER NEURAL NETWORKS C.L. Giles Air Force Office of Scientific Research, Bolling AFB, DC 20332 R.D. Griffin Naval Research Laboratory, Washington, DC 20375-5000 T. Maxwell Sachs-Freeman Associates, Landover, MD 20785 ABSTRACT We describe a method of constructing higher-order neural networks that respond invariantly under geometric transformations on the input space. By requiring each unit to satisfy a set of constraints on the interconnection weights, a particular structure is imposed on the network. A network built using such an architecture maintains its invariant performance independent of the values the weights assume, of the learning rules used, and of the form of the nonlinearities in the network. The invariance exhibited by a firstorder network is usually of a trivial sort, e.g., responding only to the average input in the case of translation invariance, whereas higher-order networks can perform useful functions and still exhibit the invariance. We derive the weight constraints for translation, rotation, scale, and several combinations of these transformations, and report results of simulation studies.

In this article we consider two aspects of computation with neural networks. Firstly we consider the problem of the complexity of the network required to compute classes of specified (structured) functions. We give a brief overview of basic known complexity theorems for readers familiar with neural network models but less familiar with circuit complexity theories. We argue that there is considerable computational and physiological justification for the thesis that shallow circuits (Le., networks with relatively few layers) are computationally more efficient. We hence concentrate on structured (as opposed to random) problems that can be computed in shallow (constant depth) circuits with a relatively few number (polynomial) of elements, and demonstrate classes of structured problems that are amenable to such low cost solutions. We discuss an allied problem-the complexity of learning-and close with some open problems and a discussion of the observed limitations of the theoretical approach. We next turn to a rigourous classification of how much a network of given structure can do; i.e., the computational capacity of a given construct.

Patterns of activity over real neural structures are known to exhibit timedependent behavior. It would seem that the brain may be capable of utilizing temporal behavior of activity in neural networks as a way of performing functions which cannot otherwise be easily implemented. These might include the origination of sequential behavior and the recognition of time-dependent stimuli. A model is presented here which uses neuronal populations with recurrent feedback connections in an attempt to observe and describe the resulting time-dependent behavior. Shortcomings and problems inherent to this model are discussed. Current models by other researchers are reviewed and their similarities and differences discussed.

DISTRIBUTED NEURAL INFORMATION PROCESSING IN THE VESTIBULO-OCULAR SYSTEM Clifford Lau Office of Naval Research Detach ment Pasadena, CA 91106 Vicente Honrubia* UCLA Division of Head and Neck Surgery Los Angeles, CA 90024 ABSTRACT A new distributed neural information-processing model is proposed to explain the response characteristics of the vestibulo-ocular system and to reflect more accurately the latest anatomical and neurophysiological data on the vestibular afferent fibers and vestibular nuclei. In this model, head motion is sensed topographically by hair cells in the semicircular canals. Hair cell signals are then processed by multiple synapses in the primary afferent neurons which exhibit a continuum of varying dynamics. The model is an application of the concept of "multilayered" neural networks to the description of findings in the bullfrog vestibular nerve, and allows us to formulate mathematically the behavior of an assembly of neurons whose physiological characteristics vary according to their anatomical properties. INTRODUCTION Traditionally the physiological properties of individual vestibular afferent neurons have been modeled as a linear time-invariant system based on Steinhausents description of cupular motion.

This arises from the parallelism and distributed knowledge representation which gives rise to gentle degradation as faults appear. These functions are attractive to implementation in VLSI and WSI. For example, the natural fault - tolerance could be useful in silicon wafers with imperfect yield, where the network degradation is approximately proportional to the non-functioning silicon area. To cast neural networks in engineering language, a neuron is a state machine that is either "on" or "off', which in general assumes intermediate states as it switches smoothly between these extrema. The synapses weighting the signals from a transmitting neuron such that it is more or less excitatory or inhibitory to the receiving neuron. The set of synaptic weights determines the stable states and represents the learned information in a system. The neural state, VI' is related to the total neural activity stimulated by inputs to the neuron through an activation junction, F. Neural activity is the level of excitation of the neuron and the activation is the way it reacts in a response to a change in activation.

ABSTRACT Intracellular recordings in spinal cord motoneurons and cerebral cortex neurons have provided new evidence on the correlational strength of monosynaptic connections, and the relation between the shapes of postsynaptic potentials and the associated increased firing probability. In these cells, excitatory postsynaptic potentials (EPSPs) produce crosscorrelogram peaks which resemble in large part the derivative of the EPSP. Additional synaptic noise broadens the peak, but the peak area -- i.e., the number of above-chance firings triggered per EPSP -- remains proportional to the EPSP amplitude. The consequences of these data for information processing by polysynaptic connections is discussed. The effects of sequential polysynaptic links can be calculated by convolving the effects of the underlying monosynaptic connections.

As a means of probing these cerebellar mechanisms, my colleagues and I have been conducting microelectrode studies of the neural messages that flow through the intermediate division of the cerebellum and onward to limb muscles via the rubrospinal tract. We regard this cerebellorubrospinal pathway as a useful model system for studying general problems of sensorimotor integration and adaptive brain function.

Current Focus Of Learning Research Most connectionist learning algorithms may be grouped into three general catagories, commonly referred to as supenJised, unsupenJised, and reinforcement learning. Supervised learning requires the explicit participation of an intelligent teacher, usually to provide the learning system with task-relevant input-output pairs (for two recent examples, see [1,2]). Unsupervised learning, exemplified by "clustering" algorithms, are generally concerned with detecting structure in a stream of input patterns [3,4,5,6,7]. In its final state, an unsupervised learning system will typically represent the discovered structure as a set of categories representing regions of the input space, or, more generally, as a mapping from the input space into a space of lower dimension that is somehow better suited to the task at hand. In reinforcement learning, a "critic" rewards or penalizes the learning system, until the system ultimately produces the correct output in response to a given input pattern [8]. It has seemed an inevitable tradeoff that systems needing to rapidly learn specific, behaviorally useful input-output mappings must necessarily do so under the auspices of an intelligent teacher with a ready supply of task-relevant training examples. This state of affairs has seemed somewhat paradoxical, since the processes of Rerceptual and cognitive development in human infants, for example, do not depend on the moment by moment intervention of a teacher of any sort. Learning by Doing The current work has been focused on a fourth type of learning algorithm, i.e. learning-bydoing, an approach that has been very little studied from either a connectionist perspective