We define the concept of polynomial uniform convergence of relative frequencies to probabilities in the distribution-dependent context.

In many machine learning applications, one has access, not only to training data, but also to some high-level a priori knowledge about the desired behavior ofthe system. For example, it is known in advance that the output of a character recognizer should be invariant with respect to small spatial distortionsof the input images (translations, rotations, scale changes, etcetera). We have implemented a scheme that allows a network to learn the derivative ofits outputs with respect to distortion operators of our choosing. This not only reduces the learning time and the amount of training data, but also provides a powerful language for specifying what generalizations we wish the network to perform. 1 INTRODUCTION In machine learning, one very often knows more about the function to be learned than just the training data. An interesting case is when certain directional derivatives ofthe desired function are known at certain points.

The complexity of learning in shallow I-Dimensional neural networks has been shown elsewhere to be linear in the size of the network. However, when the network has a huge number of units (as cortex has) even linear time might be unacceptable. Furthermore, the algorithm that was given to achieve this time was based on a single serial processor and was biologically implausible. In this work we consider the more natural parallel model of processing and demonstrate an expected-time complexity that is constant (i.e.

In this paper we investigate an average-case model of concept learning, and give results that place the popular statistical physics and VC dimension theories of learning curve behavior in a common framework.

We present an analysis of how the generalization performance (expected test set error) relates to the expected training set error for nonlinear learning systems,such as multilayer perceptrons and radial basis functions.

Learning is posed as a problem of function estimation, for which two principles ofsolution are considered: empirical risk minimization and structural risk minimization. These two principles are applied to two different statements ofthe function estimation problem: global and local. Systematic improvements in prediction power are illustrated in application to zip-code recognition.

Many auditory theorists consider the temporal adaptation of the auditory nerve a key aspect of speech coding in the auditory periphery. Experimentswith models of auditory localization and pitch perception also suggest temporal adaptation is an important element ofpractical auditory processing. I have designed, fabricated, and successfully tested an analog integrated circuit that models many aspects of auditory nerve response, including temporal adaptation. 1. INTRODUCTION We are modeling known and proposed auditory structures in the brain using analog VLSI circuits, with the goal of making contributions both to engineering practice andbiological understanding. Computational neuroscience involves modeling biology at many levels of abstraction. The first silicon auditory models were constructed ata fairly high level of abstraction (Lyon and Mead, 1988; Lazzaro and Mead, 1989ab; Mead et al., 1991; Lyon, 1991). The functional limitations of these silicon systems have prompted a new generation of auditory neural circuits designed at a lower level of abstraction (Watts et al., 1991; Liu et -al., 1991).

We present experimental data from an analog CMOS LSI chip that implements the Herault-Jutten adaptive neural network.

Analog hardware has obvious advantages in terms of its size, speed, cost, and power consumption. Analog chip designers, however, should not feel constrained to mapping existingdigital algorithms to silicon. Many times, new algorithms must be adapted or invented to ensure efficient implementation in analog hardware. Novel analog algorithms embedded in the hardware must be simple and obey the natural constraints of physics. Much algorithm intuition can be gained from experimenting with these continuous-time nonlinear systems. For example, the algorithm described in this paper arose from experimentation with existing analog segmentation hardware. Surprisingly,many of these "analog" algorithms may prove useful even if a computer vision researcher is limited to simulating the analog hardware on a digital computer [7] .

Alan Barr Computer Graphics 350-74 California Institute of Technology Pasadena, CA 91125 Abstract We use constrained optimization to select operating parameters for two circuits: a simple 3-transistor square root circuit, and an analog VLSI artificial cochlea. This automated method uses computer controlled measurement andtest equipment to choose chip parameters which minimize the difference between the actual circuit's behavior and a specified goal behavior. Choosing the proper circuit parameters is important to compensate formanufacturing deviations or adjust circuit performance within a certain range. As biologically-motivated analog VLSI circuits become increasingly complex, implying more parameters, setting these parameters by hand will become more cumbersome. Thus an automated parameter setting method can be of great value [Fleischer 90].