This paper addresses the issue of numerical computation in machine learning domains based on similarity metrics, such as kernel methods, spectral techniques and Gaussian processes. It presents a general solution strategybased on Krylov subspace iteration and fast N-body learning methods. The experiments show significant gains in computation and storage on datasets arising in image segmentation, object detection and dimensionality reduction. The paper also presents theoretical bounds on the stability of these methods.

Experimental data indicate that norepinephrine is critically involved in aspects of vigilance and attention. Previously, we considered the function ofthis neuromodulatory system on a time scale of minutes and longer, and suggested that it signals global uncertainty arising from gross changes in environmental contingencies. However, norepinephrine is also known to be activated phasically by familiar stimuli in welllearned tasks.Here, we extend our uncertainty-based treatment of norepinephrine tothis phasic mode, proposing that it is involved in the detection and reaction to state uncertainty within a task. This role of norepinephrine canbe understood through the metaphor of neural interrupts.

Neurons can have rapidly changing spike train statistics dictated by the underlying network excitability or behavioural state of an animal. To estimate the time course of such state dynamics from single-or multiple neuronrecordings, we have developed an algorithm that maximizes the likelihood of observed spike trains by optimizing the state lifetimes and the state-conditional interspike-interval (ISI) distributions. Our nonparametric algorithmis free of time-binning and spike-counting problems and has the computational complexity of a Mixed-state Markov Model operating on a state sequence of length equal to the total number ofrecorded spikes. As an example, we fit a two-state model to paired recordings of premotor neurons in the sleeping songbird. We find that the two state-conditional ISI functions are highly similar to the ones measured duringwaking and singing, respectively.

We initiate the study of learning from multiple sources of limited data, each of which may be corrupted at a different rate. We develop a complete theoryof which data sources should be used for two fundamental problems: estimating the bias of a coin, and learning a classifier in the presence of label noise. In both cases, efficient algorithms are provided for computing the optimal subset of data.

A dynamic texture is a video model that treats a video as a sample from a spatiotemporal stochastic process, specifically a linear dynamical system. Oneproblem associated with the dynamic texture is that it cannot model video where there are multiple regions of distinct motion. In this work, we introduce the layered dynamic texture model, which addresses this problem. We also introduce a variant of the model, and present the EM algorithm for learning each of the models. Finally, we demonstrate the efficacy of the proposed model for the tasks of segmentation and synthesis ofvideo.

The principle is illustrated in Figure 1 (a).

This paper presents a rigorous statistical analysis characterizing regimes in which active learning significantly outperforms classical passive learning. Activelearning algorithms are able to make queries or select sample locations in an online fashion, depending on the results of the previous queries. In some regimes, this extra flexibility leads to significantly faster rates of error decay than those possible in classical passive learning settings. Thenature of these regimes is explored by studying fundamental performance limits of active and passive learning in two illustrative nonparametric function classes. In addition to examining the theoretical potentialof active learning, this paper describes a practical algorithm capable of exploiting the extra flexibility of the active setting and provably improvingupon the classical passive techniques. Our active learning theory and methods show promise in a number of applications, including field estimation using wireless sensor networks and fault line detection.

We present a new kernel method for extracting semantic relations between entitiesin natural language text, based on a generalization of subsequence kernels.This kernel uses three types of subsequence patterns that are typically employed in natural language to assert relationships between two entities. Experiments on extracting protein interactions from biomedical corpora and top-level relations from newspaper corpora demonstrate the advantages of this approach.

A model of bottom-up overt attention is proposed based on the principle of maximizing information sampled from a scene. The proposed operation isbased on Shannon's self-information measure and is achieved in a neural circuit, which is demonstrated as having close ties with the circuitry existentin the primate visual cortex. It is further shown that the proposed saliency measure may be extended to address issues that currently eludeexplanation in the domain of saliency based models. Results on natural images are compared with experimental eye tracking data revealing theefficacy of the model in predicting the deployment of overt attention as compared with existing efforts. 1 Introduction There has long been interest in the nature of eye movements and fixation behavior following earlystudies by Buswell [I] and Yarbus [2]. However, a complete description of the mechanisms underlying these peculiar fixation patterns remains elusive.

Topic models, such as latent Dirichlet allocation (LDA), can be useful tools for the statistical analysis of document collections and other discrete data.The LDA model assumes that the words of each document arise from a mixture of topics, each of which is a distribution over the vocabulary. Alimitation of LDA is the inability to model topic correlation even though, for example, a document about genetics is more likely to also be about disease than x-ray astronomy. This limitation stems from the use of the Dirichlet distribution to model the variability among the topic proportions. In this paper we develop the correlated topic model (CTM), where the topic proportions exhibit correlation via the logistic normal distribution [1]. We derive a mean-field variational inference algorithm forapproximate posterior inference in this model, which is complicated bythe fact that the logistic normal is not conjugate to the multinomial. The CTM gives a better fit than LDA on a collection of OCRed articles from the journal Science. Furthermore, the CTM provides a natural wayof visualizing and exploring this and other unstructured data sets.