Industry
Generalization by Weight-Elimination with Application to Forecasting
Weigend, Andreas S., Rumelhart, David E., Huberman, Bernardo A.
Inspired by the information theoretic idea of minimum description length, we add a term to the back propagation cost function that penalizes network complexity. We give the details of the procedure, called weight-elimination, describe its dynamics, and clarify the meaning of the parameters involved. From a Bayesian perspective, the complexity term can be usefully interpreted as an assumption about prior distribution of the weights. We use this procedure to predict the sunspot time series and the notoriously noisy series of currency exchange rates. 1 INTRODUCTION Learning procedures for connectionist networks are essentially statistical devices for performing inductive inference. There is a tradeoff between two goals: on the one hand, we want such devices to be as general as possible so that they are able to learn a broad range of problems.
A VLSI Neural Network for Color Constancy
Moore, Andrew W., Allman, John, Fox, Geoffrey, Goodman, Rodney
A system for color correction has been designed, built, and tested successfully; the essential components are three custom chips built using subthreshold analog CMOS VLSI. The system, based on Land's Retinex theory of color constancy, produces colors similar in many respects to those produced by the visual system. Resistive grids implemented in analog VLSI perform the smoothing operation central to the algorithm at video rates. With the electronic system, the strengths and weaknesses of the algorithm are explored.
A Novel Approach to Prediction of the 3-Dimensional Structures of Protein Backbones by Neural Networks
Fredholm, Henrik, Bohr, Henrik, Bohr, Jakob, Brunak, Sรธren, Cotterill, Rodney M. J., Lautrup, Benny, Petersen, Steffen B.
Since Kendrew & Perutz solved the first protein structures, myoglobin and hemoglobin, and explained from the discovered structures how these proteins perform their function, it has been widely recognized that protein function is intimately linked with protein structure[l]. Within the last two decades X-ray crystallographers have solved the 3-dimensional (3D) structures of a steadily increasing number of proteins in the crystalline state, and recently 2D-NMR spectroscopy has emerged as an alternative method for small proteins in solution. Today approximately three hundred 3D structures have been solved by these methods, although only about half of them can be considered as truly different, and only around a hundred of them are solved at high resolution (that is, less than 2A). The number of protein sequences known today is well over 20,000, and this number seems to be growing at least one order of magnitude faster than the number of known 3D protein structures. Obviously, it is of great importance to develop tools that can predict structural aspects of proteins on the basis of knowledge acquired from known 3D structures.
VLSI Implementations of Learning and Memory Systems: A Review
ABSTRACT A large number of VLSI implementations of neural network models have been reported. The diversity of these implementations is noteworthy. This paper attempts to put a group of representative VLSI implementations in perspective by comparing and contrasting them. Design tradeoffs are discussed and some suggestions forthe direction of future implementation efforts are made. IMPLEMENTATION Changing the way information is represented can be beneficial.
Neural Network Application to Diagnostics and Control of Vehicle Control Systems
Diagnosis of faults in complex, real-time control systems is a complicated task that has resisted solution by traditional methods. We have shown that neural networks can be successfully employed to diagnose faults in digitally controlled powertrain systems. This paper discusses the means we use to develop the appropriate databases for training and testing in order to select the optimum network architectures and to provide reasonable estimates of the classification accuracy of these networks on new samples of data.
An Analog VLSI Splining Network
Schwartz, Daniel B., Samalam, Vijay K.
We have produced a VLSI circuit capable of learning to approximate arbitrary smooth of a single variable using a technique closely related to splines. The circuit effectively has 512 knots space on a uniform grid and has full support for learning. The circuit also can be used to approximate multi-variable functions as sum of splines. An interesting, and as of yet, nearly untapped set of applications for VLSI implementation of neural network learning systems can be found in adaptive control and nonlinear signal processing. In most such applications, the learning task consists of approximating a real function of a small number of continuous variables from discrete data points.
A four neuron circuit accounts for change sensitive inhibition in salamander retina
Teeters, Jeffrey L., Eeckman, Frank H., Werblin, Frank S.
In salamander retina, the response of On-Off ganglion cells to a central flash is reduced by movement in the receptive field surround. Through computer simulation of a 2-D model which takes into account their anatomical and physiological properties, we show that interactions between four neuron types (two bipolar and two amacrine) may be responsible for the generation and lateral conductance of this change sensitive inhibition. The model shows that the four neuron circuit can account for previously observed movement sensitive reductions in ganglion cell sensitivity and allows visualization and prediction of the spatiotemporal pattern of activity in change sensitive retinal cells.
Exploratory Feature Extraction in Speech Signals
A novel unsupervised neural network for dimensionality reduction which seeks directions emphasizing multimodality is presented, and its connection to exploratory projection pursuit methods is discussed. This leads to a new statistical insight to the synaptic modification equations governing learning in Bienenstock, Cooper, and Munro (BCM) neurons (1982). The importance of a dimensionality reduction principle based solely on distinguishing features, is demonstrated using a linguistically motivated phoneme recognition experiment, and compared with feature extraction using back-propagation network. 1 Introduction Due to the curse of dimensionality (Bellman, 1961) it is desirable to extract features from a high dimensional data space before attempting a classification. How to perform this feature extraction/dimensionality reduction is not that clear. A first simplification is to consider only features defined by linear (or semi-linear) projections of high dimensional data. This class of features is used in projection pursuit methods (see review in Huber, 1985). Even after this simplification, it is still difficult to characterize what interesting projections are, although it is easy to point at projections that are uninteresting. A statement that has recently been made precise by Diaconis and Freedman (1984) says that for most high-dimensional clouds, most low-dimensional projections are approximately normal. This finding suggests that the important information in the data is conveyed in those directions whose single dimensional projected distribution is far from Gaussian, especially at the center of the distribution.
Relaxation Networks for Large Supervised Learning Problems
Alspector, Joshua, Allen, Robert B., Jayakumar, Anthony, Zeppenfeld, Torsten, Meir, Ronny
Feedback connections are required so that the teacher signal on the output neurons can modify weights during supervised learning. Relaxation methods are needed for learning static patterns with full-time feedback connections. Feedback network learning techniques have not achieved wide popularity because of the still greater computational efficiency of back-propagation. We show by simulation that relaxation networks of the kind we are implementing in VLSI are capable of learning large problems just like back-propagation networks. A microchip incorporates deterministic mean-field theory learning as well as stochastic Boltzmann learning. A multiple-chip electronic system implementing these networks will make high-speed parallel learning in them feasible in the future.