Genre
What Size Net Gives Valid Generalization?
Baum, Eric B., Haussler, David
We address the question of when a network can be expected to generalize from m random training examples chosen from some arbitrary probability distribution, assuming that future test examples are drawn from the same distribution. Among our results are the following bounds on appropriate sample vs. network size.
Adaptive Neural Net Preprocessing for Signal Detection in Non-Gaussian Noise
Lippmann, Richard P., Beckman, Paul
A nonlinearity is required before matched filtering in mInimum error receivers when additive noise is present which is impulsive and highly non-Gaussian. Experiments were performed to determine whether the correct clipping nonlinearity could be provided by a single-input singleoutput multi-layer perceptron trained with back propagation. It was found that a multi-layer perceptron with one input and output node, 20 nodes in the first hidden layer, and 5 nodes in the second hidden layer could be trained to provide a clipping nonlinearity with fewer than 5,000 presentations of noiseless and corrupted waveform samples. A network trained at a relatively high signal-to-noise (SIN) ratio and then used as a front end for a linear matched filter detector greatly reduced the probability of error. The clipping nonlinearity formed by this network was similar to that used in current receivers designed for impulsive noise and provided similar substantial improvements in performance.
Training a 3-Node Neural Network is NP-Complete
Blum, Avrim, Rivest, Ronald L.
We consider a 2-layer, 3-node, n-input neural network whose nodes compute linear threshold functions of their inputs. We show that it is NPcomplete to decide whether there exist weights and thresholds for the three nodes of this network so that it will produce output consistent with a given set of training examples. We extend the result to other simple networks. This result suggests that those looking for perfect training algorithms cannot escape inherent computational difficulties just by considering only simple or very regular networks. It also suggests the importance, given a training problem, of finding an appropriate network and input encoding for that problem. It is left as an open problem to extend our result to nodes with nonlinear functions such as sigmoids.
Dynamics of Analog Neural Networks with Time Delay
Marcus, Charles M., Westervelt, R. M.
A time delay in the response of the neurons in a network can induce sustained oscillation and chaos. We present a stability criterion based on local stability analysis to prevent sustained oscillation in symmetric delay networks, and show an example of chaotic dynamics in a non-symmetric delay network.
Self Organizing Neural Networks for the Identification Problem
Tenorio, Manoel Fernando, Lee, Wei-Tsih
This work introduces a new method called Self Organizing Neural Network (SONN) algorithm and demonstrates its use in a system identification task. The algorithm constructs the network, chooses the neuron functions, and adjusts the weights. It is compared to the Back-Propagation algorithm in the identification of the chaotic time series. The results shows that SONN constructs a simpler, more accurate model.
Neural Networks that Learn to Discriminate Similar Kanji Characters
Mori, Yoshihiro, Yokosawa, Kazuhiko
Yoshihiro Morl Kazuhiko Yokosawa ATR Auditory and Visual Perception Research Laboratories 2-1-61 Shiromi Higashiku Osaka 540 Japan ABSTRACT A neural network is applied to the problem of recognizing Kanji characters. The recognition accuracy was higher than that of conventional methods. An analysis of connection weights showed that trained networks can discern the hierarchical structure of Kanji characters. This strategy of trained networks makes high recognition accuracy possible. Our results suggest that neural networks are very effective for Kanji character recognition. 1 INTRODUCTION Neural networks are applied to recognition tasks in many fields.
Neural Control of Sensory Acquisition: The Vestibulo-Ocular Reflex
Paulin, Michael G., Nelson, Mark E., Bower, James M.
In this paper we explore this idea by examining the function a simple cerebellar-related behavior, the vestibula-ocular reflex or VOR, in which eye movements are generated to minimize image slip on the retina during rapid head movements. Considering this system from the point of view of statistical estimation theory, our results suggest of the VOR, often regarded as a static orthat the transfer function slowly modifiable feature of the system, should actually be continuously and rapidly changed during head movements. We further suggest that these changes are under the direct control of the cerebellar cortex and propose experiments to test this hypothesis.
Training a 3-Node Neural Network is NP-Complete
Blum, Avrim, Rivest, Ronald L.
We consider a 2-layer, 3-node, n-input neural network whose nodes compute linear threshold functions of their inputs. We show that it is NPcomplete to decide whether there exist weights and thresholds for the three nodes of this network so that it will produce output consistent witha given set of training examples. We extend the result to other simple networks. This result suggests that those looking for perfect training algorithms cannot escape inherent computational difficulties just by considering only simple or very regular networks. It also suggests the importance, given a training problem, of finding an appropriate network and input encoding for that problem. It is left as an open problem to extend our result to nodes with nonlinear functions such as sigmoids.
Adaptive Neural Net Preprocessing for Signal Detection in Non-Gaussian Noise
Lippmann, Richard P., Beckman, Paul
A nonlinearity is required before matched filtering in mInimum error receivers when additive noise is present which is impulsive and highly non-Gaussian. Experiments were performed to determine whether the correct clipping nonlinearity could be provided by a single-input singleoutput multi-layerperceptron trained with back propagation. It was found that a multi-layer perceptron with one input and output node, 20 nodes in the first hidden layer, and 5 nodes in the second hidden layer could be trained to provide a clipping nonlinearity with fewer than 5,000 presentations of noiseless and corrupted waveform samples. A network trained at a relatively high signal-to-noise (SIN) ratio and then used as a front end for a linear matched filter detector greatly reduced the probability of error. The clipping nonlinearity formed by this network was similar to that used in current receivers designed for impulsive noise and provided similar substantial improvements in performance.
Letters to the Editor
Saveland, Jim, Cohen, Paul R., Hart, David M., Howe, Adele E., Kuipers, Benjamin J.
Jim Saveland For a fire in that fuel complex to Research Forester The Phoenix project ("Trial by Fire: grow to the size indicated in the time Associate Editor, AI Application in Understanding the Design Requirements indicated would require a midflame Natural Resource Management for Agents in Complex Environments." Agriculture 3) presents very interesting work in The authors go on to state, "Firefighting Forest Service forest fire simulation. I am especially objects are also accurately Southern Forest Fire Laboratory glad to see recognition that the "realtime, simulated; for example, bulldozers Route 1, Box 182A spatially distributed, multiagent, move at a maximum speed of... 0.5 Dry Branch, GA 31020 dynamic, and unpredictable fire kph when cutting a fireline." In reality, environment" provides an excellent sustained fireline production for Editor: opportunity to explore a variety of AI bulldozers is variable (0.1 - 2.0 kph) issues, such as how complex environments depending on steepness of the slope, Mr. Saveland's letter focuses our constrain the design of intelligent vegetation, and size of the bulldozer. I hope more AI researchers Furthermore, although bulldozers are between accuracy and realism.