Country
A Mean Field Theory of Layer IV of Visual Cortex and Its Application to Artificial Neural Networks
ABSTRACT A single cell theory for the development of selectivity and ocular dominance in visual cortex has been presented previously by Bienenstock, Cooper and Munrol. This has been extended to a network applicable to layer IV of visual cortex2 . In this paper we present a mean field approximation that captures in a fairly transparent manner the qualitative, and many of the quantitative, results of the network theory. Finally, we consider the application of this theory to artificial neural networks and show that a significant reduction in architectural complexity is possible. ASINGLE LAYER NETWORK AND THE MEAN FIELD APPROXIMATION We consider a receive signals from the layer (Figure 1).
Centric Models of the Orientation Map in Primary Visual Cortex
Centric Models of the Orientation Map in Primary Visual Cortex William Baxter of Computer Science, S.U.N.Y. at Buffalo, NY 14620Department Bruce Dow Department of Physiology, S.U.N.Y. at Buffalo, NY 14620 Abstract the visual cortex of the monkey the horizontal organization of the preferredIn of orientation-selective cells follows two opposing rules: 1) neighbors tendorientations Several orientation models which satisfy these constraints are found in the spacing and the topological index of their singularities. Using the rateto differ of orientation change as a measure, the models are compared to published experimental results. Introduction It has been known for some years that there exist orientation-sensitive neurons in the visual cortex of cats and mOnkeysl,2. These cells react to highly specific patterns of light occurring in narrowly circumscribed regiOns of the visual field, i.e., the cell's receptive field. The best patterns for such cells are typically not diffuse levels of but elongated bars or edges oriented at specific angles.
Speech Recognition Experiments with Perceptrons
This paper looks at two more difficult vocabularies, the alphabetic E-set and a set of polysyllabic words. The E-set is difficult because it contains weak discriminants and polysyllables are difficult because of timing variation. Polysyllabic word recognition is aided by a time pre-alignment technique based on dynamic programming andE-set recognition is improved by focusing attention. Recognition accuracies are better than 98% for both vocabularies when implemented with a single layer perceptron. INTRODUCTION Artificial neural networks perform well on simple pattern recognition tasks.
Generalization of Back propagation to Recurrent and Higher Order Neural Networks
Fernando J. Pineda Applied Physics Laboratory, Johns Hopkins University Johns Hopkins Rd., Laurel MD 20707 Abstract A general method for deriving backpropagation algorithms for networks with recurrent and higher order networks is introduced. The propagation of activation in these networks is determined by dissipative differential equations. The error signal is backpropagated by integrating an associated differential equation. The method is introduced by applying it to the recurrent generalization of the feedforward backpropagation network. The method is extended to the case of higher order networks and to a constrained dynamical system for training a content addressable memory. The essential feature of the adaptive algorithms is that adaptive equation has a simple outer product form.
Mathematical Analysis of Learning Behavior of Neuronal Models
Cheung, John Y., Omidvar, Massoud
Please address all further correspondence to: John Y. Cheung School of EECS 202 W. Boyd, CEC 219 Norman, OK 73019 (405)325-4721 November,1987 American Institute of Physics 1988 165 MATHEMATICAL ANALYSIS OF LEARNING BEHAVIOR OF NEURONAL MODELS John Y. Cheung and Massoud Omidvar School of Electrical Engineering and Computer Science ABSTRACT In this paper, we wish to analyze the convergence behavior of a number of neuronal plasticity models. Recent neurophysiological research suggests that the neuronal behavior is adaptive. In particular, memory stored within a neuron is associated with the synaptic weights which are varied or adjusted to achieve learning. A number of adaptive neuronal models have been proposed in the literature. Three specific models will be analyzed in this paper, specifically the Hebb model, the Sutton-Barto model, and the most recent trace model.
Introduction to a System for Implementing Neural Net Connections on SIMD Architectures
TheSIMD model of parallel computation is chosen, because systems of this type can be built with large numbers of processing elements. However, such systems are not naturally suited to generalized communication. A method is proposed that allows an implementation of neural network connections on massively parallel SIMD architectures. The key to this system is an algorithm that allows the formation of arbitrary connections between the "neurons". A feature is the ability to add new connections quickly.
On Properties of Networks of Neuron-Like Elements
Baldi, Pierre, Venkatesh, Santosh S.
In this article we consider two aspects of computation with neural networks. Firstly we consider the problem of the complexity of the network required to compute classes of specified (structured) functions. We give a brief overview of basic known complexity theoremsfor readers familiar with neural network models but less familiar with circuit complexity theories. We argue that there is considerable computational and physiological justification for the thesis that shallow circuits (Le., networks with relatively few layers) are computationally more efficient. We hence concentrate on structured (as opposed to random) problems that can be computed in shallow (constant depth)circuits with a relatively few number (polynomial) of elements, and demonstrate classes of structured problems that are amenable to such low cost solutions. Wediscuss an allied problem-the complexity of learning-and close with some open problems and a discussion of the observed limitations of the theoretical approach. Wenext turn to a rigourous classification of how much a network of given structure can do; i.e., the computational capacity of a given construct.
Presynaptic Neural Information Processing
Current knowledge about the activity dependence of the firing threshold, the conditions required for conduction failure, and the similarity of nodes along a single axon will be reviewed. An electronic circuit model for a site of low conduction safety in an axon will be presented. In response to single frequency stimulation the electronic circuit acts as a lowpass filter. I. INTRODUCTION The axon is often modeled as a wire which imposes a fixed delay on a propagating signal. Using this model, neural information processing is performed by synaptically sum m ing weighted contributions of the outputs from other neurons.