This paper proposes a means of using the knowledge in a network to determine the functionality or relevance of individual units, both for the purpose of understanding the network's behavior and improving its performance. The basic idea is to iteratively train the network to a certain performancecriterion, compute a measure of relevance that identifies whichinput or hidden units are most critical to performance, and automatically trim the least relevant units. This skeletonization technique canbe used to simplify networks by eliminating units that convey redundant information; to improve learning performance by first learning with spare hidden units and then trimming the unnecessary ones away, thereby constraining generalization; and to understand the behavior of networks in terms of minimal "rules."

I have previously proposed [Linsker, 1987, 1988] a principle of "maximum information preservation," also called the "infomax" principle, that may account for certain aspects of the organization of a layered perceptual network. The principle applies to a layer L of cells (which may be the input layer or an intermediate layer of the network) that provides input to a next layer M. The mapping of the input signal vector L onto an output signal vector M, f:L M, is characterized by a conditional probability density function ("pdf") p(MI L).

A pool of handwritten signatures is used to train a neural network forthe task of deciding whether or not a given signature is a forgery. The network is a feedforward net, with a binary image as input. There is a hidden layer, with a single unit output layer. The weights are adjusted according to the backpropagation algorithm. The signatures are entered into a C software program through the use of a Datacopy Electronic Digitizing Camera. The binary signatures arenormalized and centered. The performance is examined as a function of the training set and network structure. The best scores are on the order of 2% true signature rejection with 2-4% false signature acceptance.

A new optimization strategy, Mean Field Annealing, is presented. Its application to MAP restoration of noisy range images is derived and experimentally verified.

We address the question of when a network can be expected to generalize from m random training examples chosen from some arbitrary probabilitydistribution, assuming that future test examples are drawn from the same distribution. Among our results are the following bounds on appropriate sample vs. network size.

David Rogers Research Institute for Advanced Computer Science MS 230-5, NASA Ames Research Center Moffett Field, CA 94035 ABSTRACT A new viewpoint of the processing performed by Kanerva's sparse distributed memory (SDM) is presented. In conditions of near-or over-capacity, where the associative-memory behavior of the model breaksdown, the processing performed by the model can be interpreted asthat of a statistical predictor. Mathematical results are presented which serve as the framework for a new statistical viewpoint ofsparse distributed memory and for which the standard formulation ofSDM is a special case. This viewpoint suggests possible enhancements to the SDM model, including a procedure for improving the predictiveness of the system based on Holland's work with'Genetic Algorithms', and a method for improving the capacity of SDM even when used as an associative memory. OVERVIEW This work is the result of studies involving two seemingly separate topics that proved to share a common framework. The fIrst topic, statistical prediction, is the task of associating extremely large perceptual state vectors with future events.

The issues of scaling and generalization have emerged as key issues in current studies of supervised learning from examples in neural networks. Questions such as how many training patterns and training cycles are needed for a problem of a given size and difficulty, how to represent the inllUh and how to choose useful training exemplars, are of considerable theoretical and practical importance. Several intuitive rules of thumb have been obtained from empirical studies, but as yet there are few rigorous results.In this paper we summarize a study Qf generalization in the simplest possible case-perceptron networks learning linearly separable functions.The task chosen was the majority function (i.e. return a 1 if a majority of the input units are on), a predicate with a number ofuseful properties. We find that many aspects of.generalization in multilayer networks learning large, difficult tasks are reproduced in this simple domain, in which concrete numerical results and even some analytic understanding can be achieved.

ABSTRACT This report describes the design of a programmable general purpose analog neural computer and simulator. It is intended primarily for real-world real-time computations such as analysis of visual or acoustical patterns, robotics and the development of special purpose neural nets. The machine is scalable and composed of interconnected modules containing arrays ofneurons, modifiable synapses and switches. It runs entirely in analog mode but connection architecture, synaptic gains and time constants as well as neuron parameters are set digitally. Each neuron has a limited number of inputs and can be connected to any but not all other neurons.

This paper is concerced with the use of error back-propagation in phonetic classification. Our objective is to investigate the basic characteristicsof back-propagation, and study how the framework ofmulti-layer perceptrons can be exploited in phonetic recognition.

Phillip Alvelda, A. Miguel San Martin The Jet Propulsion Laboratory, California Institute of Technology, Pasadena, Ca. 91109 ABSTRACT Currently, the most complex spacecraft attitude determination and control tasks are ultimately governed by ground-based systems and personnel. Conventional on-board systems face severe computational bottlenecks introduced by serial microprocessors operating on inherently parallel problems. New computer architectures based on the anatomy of the human brain seem to promise high speed and fault-tolerant solutions to the limitations of serial processing. INTRODUCTION By design, a conventional on-board microprocessor can perform only one comparison or calculation at a time. Image or pattern recognition problems involving large template sets and high resolution can require an astronomical number of comparisons to a given database.