Deep Learning
A Recurrent Neural Network Model of Velocity Storage in the Vestibulo-Ocular Reflex
A three-layered neural network model was used to explore the organization of the vestibulo-ocular reflex (VOR). The dynamic model was trained using recurrent back-propagation to produce compensatory, long duration eye muscle motoneuron outputs in response to short duration vestibular afferent head velocity inputs. The network learned to produce this response prolongation, known as velocity storage, by developing complex, lateral inhibitory interactions among the interneurons. These had the low baseline, long time constant, rectified and skewed responses that are characteristic of real VOR interneurons. The model suggests that all of these features are interrelated and result from lateral inhibition.
Simulation of the Neocognitron on a CCD Parallel Processing Architecture
Chuang, Michael L., Chiang, Alice M.
The neocognitron is a neural network for pattern recognition and feature extraction. An analog CCD parallel processing architecture developed at Lincoln Laboratory is particularly well suited to the computational requirements ofshared-weight networks such as the neocognitron, and implementation of the neocognitron using the CCD architecture was simulated. A modification to the neocognitron training procedure, which improves network performance under the limited arithmetic precision that would be imposed by the CCD architecture, is presented.
Time-Sequential Self-Organization of Hierarchical Neural Networks
Silverman, Ronald H., Noetzel, Andrew S.
TIME-SEQUENTIAL SELF-ORGANIZATION OF HIERARCHICAL NEURAL NETWORKS Ronald H. Silverman Cornell University Medical College, New York, NY 10021 Andrew S. Noetzel polytechnic University, Brooklyn, NY 11201 ABSTRACT Self-organization of multi-layered networks can be realized by time-sequential organization of successive neural layers. Lateral inhibition operating in the surround of firing cells in each layer provides for unsupervised capture of excitation patterns presented by the previous layer. By presenting patterns of increasing complexity, in coordination with network selforganization, higher levels of the hierarchy capture concepts implicit in the pattern set. INTRODUCTION A fundamental difficulty in self-organization of hierarchical, multi-layered, networks of simple neuron-like cells is the determination of the direction of adjustment of synaptic link weights between neural layers not directly connected to input or output patterns. Several different approaches have been used to address this problem.
Time-Sequential Self-Organization of Hierarchical Neural Networks
Silverman, Ronald H., Noetzel, Andrew S.
TIME-SEQUENTIAL SELF-ORGANIZATION OF HIERARCHICAL NEURAL NETWORKS Ronald H. Silverman Cornell University Medical College, New York, NY 10021 Andrew S. Noetzel polytechnic University, Brooklyn, NY 11201 ABSTRACT Self-organization of multi-layered networks can be realized by time-sequential organization of successive neural layers. Lateral inhibition operating in the surround of firing cells in each layer provides for unsupervised capture of excitation patterns presented by the previous layer. By presenting patterns of increasing complexity, in coordination with network selforganization, higher levels of the hierarchy capture concepts implicit in the pattern set. INTRODUCTION A fundamental difficulty in self-organization of hierarchical, multi-layered, networks of simple neuron-like cells is the determination of the direction of adjustment of synaptic link weights between neural layers not directly connected to input or output patterns. Several different approaches have been used to address this problem.
Hierarchical Learning Control - An Approach with Neuron-Like Associative Memories
ABSTRACT Advances in brain theory need two complementary approaches: Analytical investigations by in situ measurements and as well synthetic modelling supported by computer simulations to generate suggestive hypothesis on purposeful structures in the neural tissue. In this paper research of the second line is described: Starting from a neurophysiologically inspired model of stimulusresponse (SR) and/or associative memorization and a psychologically motivated ministructure for basic control tasks, preconditions and conditions are studied for cooperation of such units in a hierarchical organisation, as can be assumed to be the general layout of macrostructures in the brain. I. INTRODUCTION Theoretic modelling in brain theory is a highly speculative subject. However, it is necessary since it seems very unlikely to get a clear picture of this very complicated device by just analyzing the available measurements on sound and/or damaged brain parts only. As in general physics, one has to realize, that there are different levels of modelling: in physics stretching from the atomary level over atom assemblies till up to general behavioural models like kinematics and mechanics, in brain theory stretching from chemical reactions over electrical spikes and neuronal cell assembly cooperation till general human behaviour.
Time-Sequential Self-Organization of Hierarchical Neural Networks
Silverman, Ronald H., Noetzel, Andrew S.
Lateral inhibition operating in the surround of firing cells in each layer provides for unsupervised capture of excitation patterns presented by the previous layer. By presenting patterns of increasing complexity, in coordination with network selforganization, higherlevels of the hierarchy capture concepts implicit in the pattern set. INTRODUCTION A fundamental difficulty in self-organization of hierarchical, multi-layered, networks of simple neuron-like cells is the determination of the direction of adjustment of synaptic link weights between neural layers not directly connected to input or output patterns. Several different approaches have been used to address this problem. One is to provide teaching inputs to the cells in internal layers of the hierarchy.
Hierarchical Learning Control - An Approach with Neuron-Like Associative Memories
In this paper research of the second line is described: Starting from a neurophysiologically inspired model of stimulusresponse (SR)and/or associative memorization and a psychologically motivated ministructure for basic control tasks, preconditions and conditions are studied for cooperation of such units in a hierarchical organisation, as can be assumed to be the general layout of macrostructures in the brain. I. INTRODUCTION Theoretic modelling in brain theory is a highly speculative subject. However, it is necessary since it seems very unlikely to get a clear picture of this very complicated device by just analyzing theavailable measurements on sound and/or damaged brain parts only. As in general physics, one has to realize, that there are different levels of modelling: in physics stretching from the atomary levelover atom assemblies till up to general behavioural models like kinematics and mechanics, in brain theory stretching from chemical reactions over electrical spikes and neuronal cell assembly cooperation till general human behaviour. The research discussed in this paper is located just above the direct study of synaptic cooperation of neuronal cell assemblies as studied e. g. in /Amari 1988/. It takes into account the changes of synaptic weighting, without simulating the physical details of such changes, and makes use of a general imitation of learning situation (stimuli) - response connections for building up trainable basic control loops, which allow dynamic SR memorization and which are themsel ves elements of some more complex behavioural loops.