Recently researchers have derived formal complexity analysis of analog computation in the setting of discrete-time dynamical systems. As an empirical constrast, training recurrent neural networks (RNNs) produces self -organized systems that are realizations of analog mechanisms. Previous workshowed that a RNN can learn to process a simple context-free language (CFL) by counting. Herein, we extend that work to show that a RNN can learn a harder CFL, a simple palindrome, by organizing its resources intoa symbol-sensitive counting solution, and we provide a dynamical systemsanalysis which demonstrates how the network: can not only count, but also copy and store counting infonnation. 1 INTRODUCTION Several researchers have recently derived results in analog computation theory in the setting ofdiscrete-time dynamical systems(Siegelmann, 1994; Maass & Opren, 1997; Moore, 1996; Casey, 1996). For example, a dynamical recognizer (DR) is a discrete-time continuous dynamicalsystem with a given initial starting point and a finite set of Boolean output decision functions(pollack.

The National Aeronautics and Space Administration Jet Propulsion Laboratory (Pasadena, Calif.) has developed The chip, which has The National Aeronautics and Chester, N.Y.) to improve its ability to been licensed by automaker Ford Space Administration (NASA) Goddard match reported wage information. Motor (Dearborn, Mich.), is designed Space Flight Center (Greenbelt, The solution will help the agency to augment current vehicle on-board Md.) has developed the The Philippines (Quezon City, The process for outside scientists wanting RoyScot Trust, the asset finance arm Philippines) has adopted an intelligent to use NASA's space telescopes. of the Royal Bank of Scotland (Edinburgh, agent-based software system to The system is designed to capture and Scotland), has implemented an manage mission-critical tax processes maintain key scientific knowledge expert system-based solution to automate across The Philippines. The intelligent while it reduces common errors made the credit-underwriting process. The firm has set up a credit control management of the bureau's entire Johnson Controls (Milwaukee, Wis.), system, The turnkey expert installs and maintains. By integrating component has deployed a speech-recognition- frequent air travelers through U.S. math data with work-cell visualization based application for its frequent flier Immigration inspection in less than software, engineers can simulate customers.

To effectively generate a fast and consistent apartment construction project network, Hyundai Engineering and Construction and Korea Advanced Institute of Science and Technology developed a case- and constraint-based project-planning expert system for an apartment domain. The system, FAS-TRAK- APT, is inspired by the use of previous cases by a human expert project planner for planning a new project and the modification of these cases by the project planner using his/her knowledge of domain constraints. This large-scale, case-based, and mixed-initiative planning system, integrated with intensive constraint-based adaptation, utilizes semantic-level metaconstraints and human decisions for compensating incomplete cases imbedding specific planning knowledge. The case- and constraint-based architecture inherently supports cross-checking cases with constraints during system development and maintenance. This system has drastically reduced the time and effort required for initial project planning, improved the quality and completeness of the generated plans, and is expected to give the company the competitive advantage in contract bids for new contracts.

We present a mixture of experts (ME) approach to interpolate sparse, spatially correlated earth-science data. Kriging is an interpolation method which uses a global covariation model estimated from the data to take account of the spatial dependence in the data. Based on the close relationship between kriging and the radial basis function (RBF) network (Wan & Bone, 1996), we use a mixture of generalized RBF networks to partition the input space into statistically correlated regions and learn the local covariation model of the data in each region. Applying the ME approach to simulated and real-world data, we show that it is able to achieve good partitioning of the input space, learn the local covariation models and improve generalization.

Further im- 528 F. Leisch and K. Hornik provements should be possible based on a better understanding of the theoretical properties of resample and combine techniques. These issues are currently being investigated.

This paper investigates the stationary points of a Hebb learning rule with a sigmoid nonlinearity in it. We show mathematically that when the input has a low information content, as measured by the input's variance, this learning rule suppresses learning, that is, forces the weight vector to converge to the zero vector. When the information content exceeds a certain value, the rule will automatically begin to learn a feature in the input. Our analysis suggests that under certain conditions it is the first principal component that is learned. The weight vector length remains bounded, provided the variance of the input is finite. Simulations confirm the theoretical results derived.

It is also shown that bounds following the true learning curve can be derived from a formalism based on the density of error patterns.

Baum and Haussler [4] used these results to give sample size bounds for multi-layer threshold networks Generalization and the Size of the Weights in Neural Networks 135 that grow at least as quickly as the number of weights (see also [7]). However, for pattern classification applications the VC-bounds seem loose; neural networks often perform successfully with training sets that are considerably smaller than the number of weights. This paper shows that for classification problems on which neural networks perform well, if the weights are not too big, the size of the weights determines the generalization performance. In contrast with the function classes and algorithms considered in the VC-theory, neural networks used for binary classification problems have real-valued outputs, and learning algorithms typically attempt to minimize the squared error of the network output over a training set. As well as encouraging the correct classification, this tends to push the output away from zero and towards the target values of { -1, I}.

We present a mixture of experts (ME) approach to interpolate sparse, spatially correlated earth-science data. Kriging is an interpolation method which uses a global covariation model estimated from the data to take account of the spatial dependence in the data. Based on the close relationship between kriging and the radial basis function (RBF) network (Wan & Bone, 1996), we use a mixture of generalized RBF networks to partition the input space into statistically correlated regions and learn the local covariation model of the data in each region. Applying the ME approach to simulated and real-world data, we show that it is able to achieve good partitioning of the input space, learn the local covariation models and improve generalization.

Further im- 528 F. Leisch and K. Hornik provements should be possible based on a better understanding of the theoretical properties of resample and combine techniques. These issues are currently being investigated.