A statistical theory for overtraining is proposed. The analysis treats realizable stochastic neural networks, trained with Kullback Leibler loss in the asymptotic case. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Considering cross-validation stopping we answer the question: In what ratio the examples should be divided into training and testing sets in order to obtain the optimum performance. In the non-asymptotic region cross-validated early stopping always decreases the generalization error. Our large scale simulations done on a CM5 are in nice agreement with our analytical findings.

In this paper the problem of learning appropriate domain-specific bias is addressed. It is shown that this can be achieved by learning many related tasks from the same domain, and a theorem is given bounding the number tasks that must be learnt.

The center-surround opponent receptive field (CSRF) mechanism represents one such example. Here, analogous CSRFs are shown to be formed in an artificial neural network which learns to localize contours (edges) of the luminance difference. Furthermore, when the input pattern is corrupted by a background noise, the CSRFs of the hidden units becomes shallower and broader with decrease of the signal-to-noise ratio (SNR). The same kind of SNR-dependent plasticity is present in the CSRF of real visual neurons; in bipolar cells of the carp retina as is shown here experimentally, as well as in large monopolar cells of the fly compound eye as was described by others. Also, analogous SNRdependent plasticity is shown to be present in the biphasic flash responses (BPFR) of these artificial and biological visual systems. Thus, the spatial (CSRF) and temporal (BPFR) filtering properties with which a wide variety of creatures see the world appear to be optimized for detectability of changes in space and time. 1 INTRODUCTION A number of learning algorithms have been developed to make synthetic neural machines be trainable to function in certain optimal ways. If the brain and nervous systems that we see in nature are best answers of the evolutionary process, then one might be able to find some common'softwares' in real and artificial neural systems. This possibility is examined in this paper, with respect to a basic visual 160 S. Y ASUI, T. FURUKAWA, M. YAMADA, T. SAITO

Several regions of the rat brain contain neurons known as head-direction celis, which encode the animal's directional heading during spatial navigation. This paper presents a biophysical model of head-direction cell acti vity, which suggests that a thalamocortical circuit might compute the rat's head direction by integrating the angular velocity of the head over time. The model was implemented using the neural simulator NEURON, and makes testable predictions about the structure and function of the rat head-direction circuit.

Because of the distance between the skull and brain and their different resistivities, electroencephalographic (EEG) data collected from any point on the human scalp includes activity generated within a large brain area. This spatial smearing of EEG data by volume conduction does not involve significant time delays, however, suggesting that the Independent Component Analysis (ICA) algorithm of Bell and Sejnowski [1] is suitable for performing blind source separation on EEG data.

The existence of significant delays in sensorimotor feedback pathways has led several researchers to suggest that the cerebellum might function as a forward model of the motor plant in order to predict the sensory consequences of motor commands before actual feedback is available; e.g., (Ito, 1984; Keeler, 1990; Miall et ai., 1993). While we agree that there are many potential roles for forward models in motor control systems, as discussed, e.g., in (Wolpert et al., 1995), we present a hypothesis about how the cerebellum could participate in regulating movement in the presence of significant feedback delays without resorting to a forward model. We show how a very simplified version of the adjustable pattern generator (APG) model being developed by Houk and colleagues (Berthier et al., 1993; Houk et al., 1995) can learn to control endpoint positioning of a nonlinear spring-mass system with significant delays in both afferent and efferent pathways. Although much simpler than a multilink dynamic arm, control of this spring-mass system involves some of the challenges critical in the control of a more realistic motor system and serves to illustrate the principles we propose. Preliminary results appear in (Buckingham et al., 1995).

Selective suppression of transmission at feedback synapses during learning is proposed as a mechanism for combining associative feedback with self-organization of feed forward synapses. Experimental data demonstrates cholinergic suppression of synaptic transmission in layer I (feedback synapses), and a lack of suppression in layer IV (feedforward synapses). A network with this feature uses local rules to learn mappings which are not linearly separable. During learning, sensory stimuli and desired response are simultaneously presented as input. Feedforward connections form self-organized representations of input, while suppressed feedback connections learn the transpose of feedforward connectivity. During recall, suppression is removed, sensory input activates the self-organized representation, and activity generates the learned response.

Binaural coincidence detection is essential for the localization of external sounds and requires auditory signal processing with high temporal precision. We present an integrate-and-fire model of spike processing in the auditory pathway of the barn owl. It is shown that a temporal precision in the microsecond range can be achieved with neuronal time constants which are at least one magnitude longer. An important feature of our model is an unsupervised Hebbian learning rule which leads to a temporal fine tuning of the neuronal connections.

Various parameterizations of rotations are related through a unifying mathematical treatment, and transformations between coordinate systems are computed using the Campbell-Baker Hausdorff formula. Next, we describe Listing's law by means of the Lie algebra so(3). This enables us to demonstrate a direct connection to Donders' law, by showing that eye orientations are restricted to the quotient space 80(3)/80(2). The latter is equivalent to the sphere S2, which is exactly the space of gaze directions. Our analysis provides a mathematical framework for studying the oculomotor system and could also be extended to investigate the geometry of mUlti-joint arm movements.

The passeriformes or songbirds make up more than half of all bird species and are divided into two groups: the os cines which learn their songs and sub-oscines which do not. Oscines raised in isolation sing degraded species typical songs similar to wild song. Deafened oscines sing completely degraded songs (Konishi, 1965), while deafened sub-oscines develop normal songs (Kroodsma and Konishi, 1991) indicating that auditory feedback is crucial in oscine song learning. Innate structures in the bird brain regulate song learning. For example, song sparrows show innate preferences for their own species' songs and song structure (Marler, 1991). Innate preferences are thought to be encoded in an auditory template which limits the sounds young birds may copy. According to the auditory template hypothesis birds go through two phases during song learning, a memorization phase and a motor phase.