In Slow Feature Analysis (SFA [1]), it has been demonstrated that high-order invariant properties can be extracted by projecting inputs intoa nonlinear space and computing the slowest changing features in this space; this has been proposed as a simple general model for learning nonlinear invariances in the visual system. However, thismethod is highly constrained by the curse of dimensionality which limits it to simple theoretical simulations. This paper demonstrates that by using a different but closely-related objective function for extracting slowly varying features ([2, 3]), and then exploiting thekernel trick, this curse can be avoided. Using this new method we show that both the complex cell properties of translation invarianceand disparity coding can be learnt simultaneously from natural images when complex cells are driven by simple cells also learnt from the image. The notion of maximising an objective function based upon the temporal predictability ofoutput has been progressively applied in modelling the development of invariances in the visual system.

Adaptation is a ubiquitous neural and psychological phenomenon, with a wealth of instantiations and implications. Although a basic form of plasticity, it has, bar some notable exceptions, attracted computational theory of only one main variety. In this paper, we study adaptation from the perspective of factor analysis, a paradigmatic technique of unsupervised learning.We use factor analysis to reinterpret a standard view of adaptation, and apply our new model to some recent data on adaptation in the domain of face discrimination.

We introduce the notion of Morton-style factorial coding and illustrate how it may help understand information integration and perceptual coding inthe brain. We show that by focusing on average responses one may miss the existence of factorial coding mechanisms that become only apparent when analyzing spike count histograms. We show evidence suggesting that the classical/nonclassical receptive field organization in the cortex effectively enforces the development of Morton-style factorial codes. This may provide some cues to help understand perceptual coding inthe brain and to develop new unsupervised learning algorithms. While methods like ICA (Bell & Sejnowski, 1997) develop independent codes, in Morton-style coding the goal is to make two or more external aspects of the world become independent when conditioning on internal representations.

Here we derive optimal gain functions for minimum mean square reconstruction fromneural rate responses subjected to Poisson noise. The shape of these functions strongly depends on the length T of the time window within which spikes are counted in order to estimate the underlying firingrate. A phase transition towards pure binary encoding occurs if the maximum mean spike count becomes smaller than approximately three provided the minimum firing rate is zero. For a particular function class, we were able to prove the existence of a second-order phase transition analytically.The critical decoding time window length obtained from the analytical derivation is in precise agreement with the numerical results. We conclude that under most circumstances relevant to information processingin the brain, rate coding can be better ascribed to a binary (low-entropy) code than to the other extreme ofrich analog coding. 1 Optimal neuronal gain functions for short decoding time windows The use of action potentials (spikes) as a means of communication is the striking feature of neurons in the central nervous system.

Language acquisition is a special kind of learning problem because the outcome of learning of one generation is the input for the next. That makes it possible for languages to adapt to the particularities of the learner. In this paper, I show that this type of language change has important consequences for models of the evolution and acquisition of syntax. 1 The Language Acquisition Problem For both artificial systems and nonhuman animals, learning the syntax of natural languages is a notoriously hard problem. All healthy human infants, in contrast, learn any of the approximately 6000 human languages rapidly, accurately and spontaneously. Anyexplanation of how they accomplish this difficult task must specify the (innate) inductive bias that human infants bring to bear, and the input data that is available to them. Traditionally, the inductive bias is termed - somewhat unfortunately -"Universal Grammar", and the input data "primary linguistic data". Over the last 30 years or so, a view on the acquisition of the syntax of natural language has become popular that has put much emphasis on the innate machinery. In this view, that one can call the "Principles and Parameters" model, the Universal Grammar specifies most aspects of syntax in great detail [e.g.

The automotive industry was the first to promote the development of applications of model-based systems technology on a broad scale and, as a result, has produced some of the most advanced prototypes and products. In this article, we illustrate the features and benefits of model-based systems and qualitative modeling by prototypes and application systems that were developed in the automotive industry to support on-board diagnosis, design for diagnosability, and failure modes and effects analysis.

Aided Design of User Interfaces. (ICKEDS 2004). "Halpern presents a masterful, complete and unified account of the many ways in which the connections between logic, probability theory and commonsensical linguistic terms can be formalized. 'believed,' 'known,' 'default,' 'relevant,' "Presents a novel thesis--that the mind is a'independent,' and'preferred' are given rigorous program whose components are semantically semantical and syntactical analyses, and their meaningful modules--and explores it with a rich interrelationships explicated and exemplified. An array of evidence drawn from a variety of fields.

In general, modeling is a complex and creative task, and building qualitative models is no exception. One way of automating this task is by means of machine learning. Observed behaviors of a modeled system are used as examples for a learning algorithm that constructs a model that is consistent with the data. In this article, we review approaches to learning qualitative models, either from numeric data or qualitative observations. We describe the QUIN program that looks for qualitative patterns in numeric data and outputs the results of learning as "qualitative trees." We illustrate this using applications associated with systems control, in particular, the identification and optimization of controllers and human operator's control skill. We also review approaches that learn models in terms of qualitative differential equations.

Complex electro-mechanical products, such as high-end printers and photocopiers, are designed as families, with reusable modules put together in different manufacturable configurations, and the ability to add new modules in the field. The modules are controlled locally by software that must take into account the entire configuration. This poses two problems for the manufacturer. The first is how to make the overall control architecture adapt to, and use productively, the inclusion of particular modules. The second is to decide, at design time, whether a proposed module is a worthwhile addition to the system: will the resulting system perform enough better to outweigh the costs of including the module? This article indicates how the use of qualitative, constraint-based models provides support for solving both of these problems. This has become an accepted part of the practice of Xerox, and the control software is deployed in high-end Xerox printers.

We examine different formalisms for modeling qualitatively physical systems and their associated inferential processes that allow us to derive qualitative predictions from the models. We highlight the mathematical aspects of these processes along with their potential and limitations. The article then bridges to quantitative modeling, highlighting the benefits of qualitative reasoning-based approaches in the framework of system identification, and discusses open research issues.