Vision based mobile robot guidance has proven difficult for classical machine vision methods because of the diversity and real time constraints inherent in the task. This thesis describes a connectionist system called ALVINN (Autonomous Land Vehicle In a Neural Network) that overcomes these difficulties. ALVINN learns to guide mobile robots using the back-propagation training algorithm. Because of its ability to learn from example, ALVINN can adapt to new situations and therefore cope with the diversity of the autonomous navigation task. But real world problems like vision based mobile robot guidance presents a different set of challenges for the connectionist paradigm.

Neural networks, despite their empirically proven abilities, have been little used for the refinement of existing knowledge because this task requires a three-step process. First, knowledge must be inserted into a neural network. Second, the network must be refined. Third, the refined knowledge must be extracted from the network. We have previously described a method for the first step of this process.

Creating an optimum long-term schedule for the Hubble Space Telescope is difficult by almost any standard due to the large number of activities, many relative and absolute time constraints, prevailing uncertainties and an unusually wide range of timescales. This problem has motivated research in neural networks for scheduling. The novel concept of continuous suitaility functions defined over a continuous time domain has been developed to represent soft temporal relationships between activities. All constraints and preferences are automatically translated into the weights of an appropriately designed artificial neural network. The constraints are subject to propagation and consistency enhancement in order to increase the number of explicitly represented constraints.

A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible (1) objective comparisons between solutions using alternative network architectures, (2) objective stopping rules for network pruning or growing procedures, (3) objective choice of magnitude and type of weight decay terms or additive regularizers (for penalizing large weights, etc.), (4) a measure of the effective number of well-determined parameters in a model, (5) quantified estimates of the error bars on network parameters and on network output, and (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian "evidence" automatically embodies "Occam's razor," penalizing overflexible and overcomplex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalization ability and the Bayesian evidence is obtained.

Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perceptron and backpropagation neural learning algorithms have been performed using five large, real-world data sets. Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs.

It is important to recognize that natural selection, when viewed as a. computational paradigm for search and learning, places severe restrictions on possible

Using the tools of complexity theory, Stephen Judd develops a formal description of associative learning in connectionist networks. He rigorously exposes the computational difficulties in training neural networks and explores how certain design principles will or will not make the problems easier. Judd looks beyond the scope of any one particular learning rule, at a level above the details of neurons. There he finds new issues that arise when great numbers of neurons are employed and he offers fresh insights into design principles that could guide the construction of artificial and biological neural networks. The first part of the book describes the motivations and goals of the study and relates them to current scientific theory.

Full text available for a fee. This book is a practical guide to classification learning systems and their applications. These computer programs learn from sample data and make predictions for new cases, sometimes exceeding the performance of humans. Practical learning systems from statistical pattern recognition, neural networks, and machine learning are presented. The authors examine prominent methods from each area, using an engineering approach and taking the practitioner's viewpoint. Intuitive explanations with a minimum of mathematics make the material accessible to anyone--regardless of experience or special interests. The underlying concepts of the learning methods are discussed with fully worked-out examples: their strengths and weaknesses, and the estimation of their future performance on specific applications. Throughout, the authors offer their own recommendations for selecting and applying learning methods such as linear discriminants, back-propagation neural networks, or decision trees. Learning systems are then contrasted with their rule-based counterparts from expert systems.Morgan Kaufmann, 1990

A class of connectionist networks is described that has learned to play backgammon at an intermediate-to-advanced level. The networks were trained by back-propagation learning on a large set of sample positions evaluated by a human expert. In actual match play against humans and conventional computer programs, the networks have demonstrated substantial ability to generalize on the basis of expert knowledge of the game. This is possibly the most complex domain yet studied with connectionist learning. New techniques were needed to overcome problems due to the scale and complexity of the task.

A major goal of research on networks of neuron-like processing units is to discover efficient learning procedures that allow these networks to construct complex internal representations of their environment. The learning procedures must be capable of modifying the connection strengths in such a way that internal units which are not part of the input or output come to represent important features of the task domain. Several interesting gradient-descent procedures have recently been discovered. Each connection computes the derivative, with respect to the connection strength, of a global measure of the error in the performance of the network. The strength is then adjusted in the direction that decreases the error.