We study the complexity problem in artificial feedforward neural networks designed to approximate real valued functions of several real variables; i.e., we estimate the number of neurons in a network required to ensure a given degree of approximation to every function in a given function class. We indicate how to construct networks with the indicated number of neurons evaluating standard activation functions. Our general theorem shows that the smoother the activation function, the better the rate of approximation. 1 INTRODUCTION The approximation capabilities of feedforward neural networks with a single hidden layer has been studied by many authors, e.g., [1, 2, 5]. In [10], we have shown that such a network using practically any nonlinear activation function can approximate any continuous function of any number of real variables on any compact set to any desired degree of accuracy. A central question in this theory is the following.

Recently, Ott, Grebogi and Yorke (OGY) [6] found an effective method to control chaotic systems to unstable fixed points by using onlysmall control forces; however, OGY's method is based on and limited to a linear theory and requires considerable knowledge of the dynamics of the system to be controlled. In this paper we use two radial basis function networks: one as a model of an unknown plant and the other as the controller. The controller is trained with a recurrent learning algorithm to minimize a novel objective function such that the controller can locate an unstable fixed point and drive the system into the fixed point with no a priori knowledge ofthe system dynamics. Our results indicate that the neural controller offers many advantages over OGY's technique.

Automated speech recognition is notoriously hard, and thus any predictive source of information and constraints that could be incorporated into a computer speech recognition system would be desirable. Humans, especially the hearing impaired, can utilize visual information - "speech reading" - for improved accuracy (Dodd & Campbell, 1987, Sanders & Goodrich, 1971). Speech reading can provide direct information about segments, phonemes, rate, speaker gender and identity, and subtle informationfor segmenting speech from background noise or multiple speakers (De Filippo & Sims, 1988, Green & Miller, 1985). Fundamental support for the use of visual information comes from the complementary natureof the visual and acoustic speech signals. Utterances that are difficult to distinguish acoustically are the easiest to distinguish.

Scope This workshop reviewed and classified the various models which have emerged from the general concept of selective attention and context dependency, and sought to identify their commonalities. It was concluded that the motivation and mechanism ofthese functional models are "efficiency" and ''factoring'', respectively. The workshop focused on computational models of selective attention and context dependency withinthe realm of neural networks. We treated only ''functional'' models; computational models of biological neural systems, and symbolic or rule-based systems were omitted from the discussion. Presentations Thomas H. Hildebrandt presented the results of his recent survey of the literature onfunctional models of selective attention and context dependency.

We developed a system for finding address blocks on mail pieces that can process four images per second. Besides locating the address block, our system also determines the writing style, handwritten or machine printed, and moreover, it measures the skew angle of the text lines and cleans noisy images. A layout analysis of all the elements present in the image is performed in order to distinguish drawings and dirt from text and to separate text of advertisement from that of the destination address. A speed of more than four images per second is obtained on a modular hardware platform, containing a board with two of the NET32K neural net chips, a SPARC2 processor board, and a board with 2 digital signal processors. The system has been tested with more than 100,000 images. Its performance depends on the quality of the images, and lies between 85% correct location in very noisy images to over 98% in cleaner images.

Based on precise anatomical data of the bee's olfactory system, we propose an investigation of the possible mechanisms of modulation and control between the two levels of olfactory information processing: the antennallobe glomeruli and the mushroom bodies. We use simplified neurons, but realistic architecture. As a first conclusion, we postulate that the feature extraction performed by the antennallobe (glomeruli and interneurons) necessitates central input from the mushroom bodies for fine tuning.

We have developed an artificial neural network based gaze tracking system which can be customized to individual users. Unlike other gaze trackers, which normally require the user to wear cumbersome headgear, or to use a chin rest to ensure head immobility, our system is entirely non-intrusive. Currently, the best intrusive gaze tracking systems are accurate to approximately 0.75degrees. In our experiments, we have been able to achieve an accuracy of 1.5 degrees, while allowing head mobility. In this paper we present an empirical analysis of the performance of a large number of artificial neuralnetwork architectures for this task.

Andrew W. Moore School of Computer Science Carnegie-Mellon University Pittsburgh, PA 15213 Abstract Parti-game is a new algorithm for learning from delayed rewards in high dimensional real-valued state-spaces. In high dimensions it is essential that learning does not explore or plan over state space uniformly. Parti-game maintains a decision-tree partitioning of state-space and applies game-theory and computational geometry techniquesto efficiently and reactively concentrate high resolution onlyon critical areas. Many simulated problems have been tested, ranging from 2-dimensional to 9-dimensional state-spaces, including mazes, path planning, nonlinear dynamics, and uncurling snakerobots in restricted spaces. In all cases, a good solution is found in less than twenty trials and a few minutes. 1 REINFORCEMENT LEARNING Reinforcement learning [Samuel, 1959, Sutton, 1984, Watkins, 1989, Barto et al., 1991] is a promising method for control systems to program and improve themselves.

Integrated Mean Squared Error (IMSE) is a version of the usual mean squared error criterion, averaged over all possible training sets of a given size. If it could be observed, it could be used to determine optimal network complexity or optimal data subsets forefficient training. We show that two common methods of cross-validating average squared error deliver unbiased estimates of IMSE, converging to IMSE with probability one. We also show that two variants of cross validation measure provide unbiased IMSE-based estimates potentially useful for selecting optimal data subsets. 1 Summary To begin, assume we are given a fixed network architecture. Let zN denote a given set of N training examples.

A.C.C. Coolen R.W. Penney D. Sherrington Dept. of Physics - Theoretical Physics University of Oxford 1 Keble Road, Oxford OXI 3NP, U.K. Abstract A simple model of coupled dynamics of fast neurons and slow interactions, modellingself-organization in recurrent neural networks, leads naturally to an effective statistical mechanics characterized by a partition function which is an average over a replicated system. This is reminiscent of the replica trick used to study spin-glasses, but with the difference that the number of replicas has a physical meaningas the ratio of two temperatures and can be varied throughout the whole range of real values. The model has interesting phaseconsequences as a function of varying this ratio and external stimuli, and can be extended to a range of other models. 1 A SIMPLE MODEL WITH FAST DYNAMIC NEURONS AND SLOW DYNAMIC INTERACTIONS As the basic archetypal model we consider a system of Ising spin neurons (J'i E {-I, I}, i E {I, ..., N}, interacting via continuous-valued symmetric interactions, Iij, which themselves evolve in response to the states of the neurons. The neurons are taken to have a stochastic field-alignment dynamics which is fast compared with the evolution rate of the interactions hj, such that on the timescale of Iii-dynamics the neurons are effectively in equilibrium according to a Boltzmann distribution, (1) 447 448 Cooien, Penney, and Sherrington where HVoj}({O"d) JijO"iO"j (2) i j and the subscript {Jij} indicates that the {Jij} are to be considered as quenched variables. In practice, several specific types of dynamics which obey detailed balance lead to the equilibrium distribution (1), such as a Markov process with single-spin flip Glauber dynamics [1]. The second term acts to limit the magnitude of hj; f3 is the characteristic inverse temperature of the interaction system. VNTJij(t) (4) where the effective Hamiltonian 11. ({ hj}) is given by 1 1 We now recognise (4) as having the form of a Langevin equation, so that the equilibrium distributionof the interaction system is given by a Boltzmann form. Z{3 (6) Coupled Dynamics of Fast Neurons and Slow Interactions 449 where n _ /j3. We may use Z as a generating functional to produce thermodynamic averagesof state variables I ( {O"d; {Jij}) in the combined system by adding suitable infinitesimal source terms to the neuron Hamiltonian (2): HP.j}({O"d) In fact, any real n is possible by tuning the ratio between the two {3's. In the formulation presented in this paper n is always nonnegative, but negative values are possible if the Hebbian rule of (3) is replaced by an anti-Hebbian form with (UiO"j) replaced by - (O"iO"j) (the case of negative n is being studied by Mezard and coworkers [7]).