A new form of the deterministic Boltzmann machine (DBM) learning procedureis presented which can efficiently train network modules todiscriminate between input vectors according to some criterion. Thenew technique directly utilizes the free energy of these "mean field modules" to represent the probability that the criterion is met, the free energy being readily manipulated by the learning procedure. Although conventional deterministic Boltzmann learning failsto extract the higher order feature of shift at a network bottleneck, combining the new mean field modules with the mutual informationobjective function rapidly produces modules that perfectly extract this important higher order feature without direct external supervision. 1 INTRODUCTION The Boltzmann machine learning procedure (Hinton and Sejnowski, 1986) can be made much more efficient by using a mean field approximation in which stochastic binary units are replaced by deterministic real-valued units (Peterson and Anderson, 1987). Deterministic Boltzmann learning can be used for "multicompletion" tasks in which the subsets of the units that are treated as input or output are varied from trial to trial (Peterson and Hartman, 1988). In this respect it resembles other learning procedures that also involve settling to a stable state (Pineda, 1987). Using the multicompletion paradigm, it should be possible to force a network to explicitly extract important higher order features of an ensemble of training vectors by forcing the network to pass the information required for correct completions through a narrow bottleneck. In back-propagation networks with two or three hidden layers, the use of bottlenecks sometimes allows the learning to explictly discover important.

A simple method for training the dynamical behavior of a neural networkis derived. It is applicable to any training problem in discrete-time networks with arbitrary feedback. The algorithm resembles back-propagation in that an error function is minimized using a gradient-based method, but the optimization is carried out in the hidden part of state space either instead of, or in addition to weight space. Computational results are presented for some simple dynamical training problems, one of which requires response to a signal 100 time steps in the past. 1 INTRODUCTION This paper presents a minimization-based algorithm for training the dynamical behavior ofa discrete-time neural network model. The central idea is to treat hidden nodes as target nodes with variable training data.

In the development of an image segmentation system for real time image processing applications, we apply the classical decision analysis paradigmby viewing image segmentation as a pixel classifica.

Selective sampling

A new concept for unsupervised learning based upon examples introduced tothe neural network is proposed. Each example is considered as an interpolation node of the velocity field in the phase space. The velocities at these nodes are selected such that all the streamlines converge to an attracting set imbedded in the subspace occupied by the cluster of examples. The synaptic interconnections are found from learning procedure providing selected field. The theory is illustrated by examples. This paper is devoted to development of a new concept for unsupervised learning based upon examples introduced to an artificial neural network.

Cascade-Correlation is a new architecture and supervised learning algorithm forartificial neural networks. Instead of just adjusting the weights in a network of fixed topology. Cascade-Correlation begins with a minimal network,then automatically trains and adds new hidden units one by one, creating a multi-layer structure. Once a new hidden unit has been added to the network, its input-side weights are frozen. This unit then becomes a permanent feature-detector in the network, available for producing outputs or for creating other, more complex feature detectors. TheCascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network .determines

We are developing a phoneme based.

With the growing interest in the practical use of neural networks, addressing the problem of customiling networks for specific applications is becoming increasingly critical.It has repeatedly been observed that different network structures and learning parameters can substantially affect performance. Such important aspects of neural network applications as generalilation, learning speed, connectivity andtolerance to network damage are strongly related to the choice of 448 Harp, Samad and Guha network architecture. Yet there are few analytic results, and few heuristics, that can help the application developer design an appropriate network. We have been investigating the use of the genetic algorithm (Goldberg, 1989; Holland, 1975) for designing application-specific neural networks (Harp, Samad and Guha, 1989ab). In our approach, the genetic algorithm is used to evolve appropriate network structures and values of learning parameters.

Linsker has reported the development of centre---surround receptive fields and oriented receptive fields in simulations of a Hebb-type equation in a linear network. The dynamics of the learning rule are analysed in terms of the eigenvectors of the covariance matrix of cell activities. Analytic and computational results for Linsker's covariance matrices, and some general theorems, lead to an explanation ofthe emergence of centre---surround and certain oriented structures. Linsker [Linsker, 1986, Linsker, 1988] has studied by simulation the evolution of weight vectors under a Hebb-type teacherless learning rule in a feed-forward linear network. The equation for the evolution of the weight vector w of a single neuron, derived by ensemble averaging the Hebbian rule over the statistics of the input patterns, is:!

The midbrain of the barn owl contains a map-like representation of sound source direction which is used to precisely orient the head toward targetsof interest. Elevation is computed from the interaural difference in sound level. We present models and computer simulations oftwo stages of level difference processing which qualitatively agree with known anatomy and physiology, and make several striking predictions. 1 INTRODUCTION