Two approaches were explored which integrate neural net classifiers with Hidden Markov Model (HMM) speech recognizers.

Analytic solutions to the information-theoretic evolution equation ofthe connection strength of a three-layer feedforward neural net for visual information processing are presented. The results are (1) the receptive fields of the feature-analysing cells correspond tothe eigenvector of the maximum eigenvalue of the Fredholm integral equation of the first kind derived from the evolution equation of the connection strength; (2) a symmetry-breaking mechanism (parity-violation) has been identified to be responsible forthe changes of the morphology of the receptive field; (3) the conditions for the formation of different morphologies are explicitly identified.

We propose a new way to construct a large-scale neural network for 3.000 handwritten Kanji characters recognition. This neural network consists of 3 parts: a collection of small-scale networks which are trained individually on a small number of Kanji characters; a network which integrates the output from the small-scale networks, and a process to facilitate the integration of these neworks. The recognition rate of the total system is comparable with those of the small-scale networks. Our results indicate that the proposed method is effective for constructing a large-scale network without loss of recognition performance.

Eight neural net and conventional pattern classifiers (Bayesianunimodal Gaussian,k-nearest neighbor, standard back-propagation, adaptive-stepsize back-propagation, hypersphere, feature-map, learning vectorquantizer, and binary decision tree) were implemented on a serial computer and compared using two speech recognition and two artificial tasks. Error rates were statistically equivalent on almost all tasks, but classifiers differed by orders of magnitude in memory requirements, training time, classification time, and ease of adaptivity. Nearest-neighbor classifiers trained rapidly but required themost memory. Tree classifiers provided rapid classification but were complex to adapt. Back-propagation classifiers typically requiredlong training times and had intermediate memory requirements. These results suggest that classifier selection should often depend more heavily on practical considerations concerning memory and computation resources, and restrictions on training and classification times than on error rate.

Spiking neurons which integrate to threshold and fire were used to study the transmission of frequency modulated (FM) signals through layered networks. Firing correlations between cells in the input layer were found to modulate the transmission of FM signals undercertain dynamical conditions. A tonic level of activity was maintained by providing each cell with a source of Poissondistributed synapticinput. When the average membrane depolarization produced by the synaptic input was sufficiently below threshold, the firing correlations between cells in the input layer could greatly amplify the signal present in subsequent layers. When the depolarization was sufficiently close to threshold, however, the firing synchrony between cells in the initial layers could no longer effect the propagation of FM signals. In this latter case, integrateand-fire neuronscould be effectively modeled by simpler analog elements governed by a linear input-output relation.

It is generally admitted that generalization performance of back-propagation networks (Rumelhart,Hinton & Williams, 1986) will depend on the relative size ofthe training data and of the trained network. By analogy to curve-fitting and for theoretical considerations,the generalization performance of the network should decrease as the size of the network and the associated number of degrees of freedom increase (Rumelhart, 1987; Denker et al., 1987; Hanson & Pratt, 1989). This paper examines the dynamics of the standard back-propagation algorithm (BP) and of a constrained back-propagation variation (CBP), designed to adapt the size of the network to the training data base. The performance, learning dynamics and the representations resulting from the two algorithms are compared.

In associative reinforcement learning, an environment generates input vectors, a learning system generates possible output vectors, and a reinforcement functioncomputes feedback signals from the input-output pairs. The task is to discover and remember input-output pairs that generate rewards. Especially difficult cases occur when rewards are rare, since the expected time for any algorithm can grow exponentially with the size of the problem. Nonetheless, if a reinforcement function possesses regularities, and a learning algorithm exploits them, learning time can be reduced below that of non-generalizing algorithms. This paper describes a neural network algorithm called complementary reinforcement back-propagation(CRBP), and reports simulation results on problems designed to offer differing opportunities for generalization.

It has been known for many years that specific regions of the working cerebralcortex display periodic variations in correlated cellular activity. While the olfactory system has been the focus of much of this work, similar behavior has recently been observed in primary visual cortex. We have developed models of both the olfactory and visual cortex which replicate the observed oscillatory properties ofthese networks. Using these models we have examined the dependence of oscillatory behavior on single cell properties and network architectures.We discuss the idea that the oscillatory events recorded from cerebral cortex may be intrinsic to the architecture of cerebral cortex as a whole, and that these rhythmic patterns may be important in coordinating neuronal activity during sensory processmg.

We describe a model that can recognize two-dimensional shapes in an unsegmented image, independent of their orientation, position, and scale. The model, called TRAFFIC, efficiently represents the structural relation between an object and each of its component features by encoding the fixed viewpoint-invariant transformation from the feature's reference frame to the object's in the weights of a connectionist network. Using a hierarchy of such transformations, with increasing complexity of features at each successive layer, the network can recognize multiple objects in parallel. An implementation ofTRAFFIC is described, along with experimental results demonstrating the network's ability to recognize constellations of stars in a viewpoint-invariant manner. 1 INTRODUCTION A key goal of machine vision is to recognize familiar objects in an unsegmented image, independent of their orientation, position, and scale. Massively parallel models have long been used for lower-level vision tasks, such as primitive feature extraction and stereo depth.

The long-term goal of our laboratory is the development of analog resistive network-based VLSI implementations of early and intermediate visionalgorithms. We demonstrate an experimental circuit for smoothing and segmenting noisy and sparse depth data using the resistive fuse and a 1-D edge-detection circuit for computing zero-crossingsusing two resistive grids with different spaceconstants. Todemonstrate the robustness of our algorithms and of the fabricated analog CMOS VLSI chips, we are mounting these circuits onto small mobile vehicles operating in a real-time, laboratory environment.