We present a methodological framework enabling a detailed description of the performance of Hopfield-like attractor neural networks (ANN) in the first two iterations. Using the Bayesian approach, we find that performance is improved when a history-based term is included in the neuron's dynamics. A further enhancement of the network's performance is achieved by judiciously choosing the censored neurons (those which become active in a given iteration) on the basis of the magnitude of their post-synaptic potentials. The contribution of biologically plausible, censored, historydependent dynamics is especially marked in conditions of low firing activity and sparse connectivity, two important characteristics of the mammalian cortex. In such networks, the performance attained is higher than the performance of two'independent' iterations, which represents an upper bound on the performance of history-independent networks.

TJ of the gradient updates is held constant (simple backpropagation).

Certain species of freshwater tropical fish, known as weakly electric fish, possess an active electric sense that allows them to detect and discriminate objects in their environment using a self-generated electric field (Bullock and Heiligenberg, 1986). They detect objects by sensing small perturbations in this electric field using an array of specialized receptors, known as electroreceptors, that cover their body surface. Weakly electric fish often live in turbid water and tend to be nocturnal. These conditions, which hinder visual perception, do not adversely affect the electric sense. Hence the electrosensory system allows these fish to navigate and capture prey in total darkness in much the same way as the sonar system of echolocating bats allows them to do the same.

We describe a model of visual word recognition that accounts for several aspects of the temporal processing of sequences of briefly presented words. The model utilizes a new representation for written words, based on dynamic time warping and multidimensional scaling. The visual input passes through cascaded perceptual, comparison, and detection stages. We describe how these dynamical processes can account for several aspects of word recognition, including repetition priming and repetition blindness.

This paper describes a technique for learning both the number of states and the topology of Hidden Markov Models from examples. The induction process starts with the most specific model consistent with the training data and generalizes by successively merging states. Both the choice of states to merge and the stopping criterion are guided by the Bayesian posterior probability. We compare our algorithm with the Baum-Welch method of estimating fixed-size models, and find that it can induce minimal HMMs from data in cases where fixed estimation does not converge or requires redundant parameters to converge. 1 INTRODUCTION AND OVERVIEW Hidden Markov Models (HMMs) are a well-studied approach to the modelling of sequence data. HMMs can be viewed as a stochastic generalization of finite-state automata, where both the transitions between states and the generation of output symbols are governed by probability distributions. HMMs have been important in speech recognition (Rabiner & Juang, 1986), cryptography, and more recently in other areas such as protein classification and alignment (Haussler, Krogh, Mian & SjOlander, 1992; Baldi, Chauvin, Hunkapiller & McClure, 1993). Practitioners have typically chosen the HMM topology by hand, so that learning the HMM from sample data means estimating only a fixed number of model parameters. The standard approach is to find a maximum likelihood (ML) or maximum a posteriori probability (MAP) estimate of the HMM parameters.

We analyze the "query by committee" algorithm, a method for filtering informative queries from a random stream of inputs. We show that if the two-member committee algorithm achieves information gain with positive lower bound, then the prediction error decreases exponentially with the number of queries. We show that, in particular, this exponential decrease holds for query learning of thresholded smooth functions.

Principal Component Analysis (PCA) is an essential technique for data compression and feature extraction, and has been widely used in statistical data analysis, communication theory, pattern recognition and image processing. In the neural network literature, a lot of studies have been made on learning rules for implementing PCA or on networks closely related to PCA (see Xu & Yuille, 1993 for a detailed reference list which contains more than 30 papers related to these issues).

Networks with local inhibition are shown to have enhanced computational performance with respect to the classical Hopfield-like networks. In particular the critical capacity of the network is increased as well as its capability to store correlated patterns. Chaotic dynamic behaviour (exponentially long transients) of the devices indicates the overloading of the associative memory. An implementation based on a programmable logic device is here presented. A 16 neurons circuit is implemented whit a XILINK 4020 device.

A parallel stochastic algorithm is investigated for error-descent learning and optimization in deterministic networks of arbitrary topology. No explicit information about internal network structure is needed. The method is based on the model-free distributed learning mechanism of Dembo and Kailath. A modified parameter update rule is proposed by which each individual parameter vector perturbation contributes a decrease in error. A substantially faster learning speed is hence allowed. Furthermore, the modified algorithm supports learning time-varying features in dynamical networks. We analyze the convergence and scaling properties of the algorithm, and present simulation results for dynamic trajectory learning in recurrent networks.

Recently, both connectionist and symbolic methods have been developed for biasing learning with prior knowledge lFu, 1989; Towell et a/., 1990; Ourston and Mooney, 1990]. Most ofthese methods revise an imperfect knowledge base (usually obtained from a domain expert) to fit a set of empirical data. Some of these methods have been successfully applied to real-world tasks, such as recognizing promoter sequences in DNA [Towell et ai., 1990; Ourston and Mooney, 1990]. The results demonstrate that revising an expert-given knowledge base produces more accurate results than learning from training data alone.