Industry
On the Generalization Ability of On-Line Learning Algorithms
Cesa-bianchi, Nicolò, Conconi, Alex, Gentile, Claudio
In this paper we show that online algorithms for classification and regression canbe naturally used to obtain hypotheses with good datadependent tailbounds on their risk. Our results are proven without requiring complicated concentration-of-measure arguments and they hold for arbitrary online learning algorithms. Furthermore, when applied to concrete online algorithms, our results yield tail bounds that in many cases are comparable or better than the best known bounds.
Motivated Reinforcement Learning
Competition between actions is based on the motivating characteristics of their consequent states in this sense. Substantial, careful, experiments reviewed in Dickinson & Balleine,12,13 into the neurobiology and psychology ofmotivation shows that this view is incomplete. In many cases, animals are faced with the choice not between many different actionsat a given state, but rather whether a single response isworth executing at all. Evidence suggests that the motivational process underlying this choice has different psychological andneural properties from that underlying action choice. We describe and model these motivational systems, and consider the way they interact.
A Bayesian Network for Real-Time Musical Accompaniment
We describe a computer system that provides a real-time musical accompanimentfor a live soloist in a piece of non-improvised music for soloist and accompaniment. A Bayesian network is developed thatrepresents the joint distribution on the times at which the solo and accompaniment notes are played, relating the two parts through a layer of hidden variables. The network is first constructed usingthe rhythmic information contained in the musical score. The network is then trained to capture the musical interpretations ofthe soloist and accompanist in an off-line rehearsal phase. During live accompaniment the learned distribution of the network is combined with a real-time analysis of the soloist's acoustic signal, performedwith a hidden Markov model, to generate a musically principledaccompaniment that respects all available sources of knowledge. A live demonstration will be provided.
Group Redundancy Measures Reveal Redundancy Reduction in the Auditory Pathway
Chechik, Gal, Globerson, Amir, Anderson, M. J., Young, E. D., Nelken, Israel, Tishby, Naftali
The way groups of auditory neurons interact to code acoustic information isinvestigated using an information theoretic approach. We develop measures of redundancy among groups of neurons, and apply them to the study of collaborative coding efficiency in two processing stations in the auditory pathway: the inferior colliculus (IC) and the primary auditory cortex (AI). Under two schemes for the coding of the acoustic content, acoustic segments coding and stimulus identity coding, we show differences both in information content and group redundancies between IC and AI neurons. These results provide for the first time a direct evidence for redundancy reduction along the ascending auditory pathway, as has been hypothesized fortheoretical considerations [Barlow 1959,2001]. The redundancy effects under the single-spikes coding scheme are significant onlyfor groups larger than ten cells, and cannot be revealed with the redundancy measures that use only pairs of cells. The results suggest that the auditory system transforms low level representations thatcontain redundancies due to the statistical structure of natural stimuli, into a representation in which cortical neurons extractrare and independent component of complex acoustic signals, that are useful for auditory scene analysis.
Online Learning with Kernels
Kivinen, Jyrki, Smola, Alex J., Williamson, Robert C.
We consider online learning in a Reproducing Kernel Hilbert Space. Our method is computationally efficient and leads to simple algorithms. In particular we derive update equations for classification, regression, and novelty detection. The inclusion of the -trick allows us to give a robust parameterization.
Information-Geometric Decomposition in Spike Analysis
Nakahara, Hiroyuki, Amari, Shun-ichi
We present an information-geometric measure to systematically investigate neuronal firing patterns, taking account not only of the second-order but also of higher-order interactions. We begin with the case of two neurons for illustration and show how to test whether or not any pairwise correlation in one period is significantly different from that in the other period. In order to test such a hypothesis ofdifferent firing rates, the correlation term needs to be singled out'orthogonally' to the firing rates, where the null hypothesis mightnot be of independent firing. This method is also shown to directly associate neural firing with behavior via their mutual information, which is decomposed into two types of information, conveyed by mean firing rate and coincident firing, respectively. Then, we show that these results, using the'orthogonal' decomposition, arenaturally extended to the case of three neurons and n neurons in general. 1 Introduction Based on the theory of hierarchical structure and related invariant decomposition of interactions by information geometry [3], the present paper briefly summarizes methods useful for systematically analyzing a population of neural firing [9].
A Rational Analysis of Cognitive Control in a Speeded Discrimination Task
Mozer, Michael C., Colagrosso, Michael D., Huber, David E.
We are interested in the mechanisms by which individuals monitor and adjust their performance of simple cognitive tasks. We model a speeded discrimination task in which individuals are asked to classify a sequence of stimuli (Jones & Braver, 2001). Response conflict arises when one stimulus class is infrequent relative to another, resulting in more errors and slower reaction times for the infrequent class. How do control processes modulatebehavior based on the relative class frequencies? We explain performance from a rational perspective that casts the goal of individuals as minimizing a cost that depends both on error rate and reaction time.With two additional assumptions of rationality--that class prior probabilities are accurately estimated and that inference is optimal subject to limitations on rate of information transmission--we obtain a good fit to overall RT and error data, as well as trial-by-trial variations in performance.
A Natural Policy Gradient
Sham Kakade Gatsby Computational Neuroscience Unit 17 Queen Square, London, UK WC1N 3AR http://www.gatsby.ucl.ac.uk sham@gatsby.ucl.ac.uk Abstract We provide a natural gradient method that represents the steepest descent direction based on the underlying structure of the parameter space.Although gradient methods cannot make large changes in the values of the parameters, we show that the natural gradient ismoving toward choosing a greedy optimal action rather than just a better action. These greedy optimal actions are those that would be chosen under one improvement step of policy iteration with approximate, compatible value functions, as defined by Sutton etal. We then show drastic performance improvements in simple MDPs and in the more challenging MDP of Tetris. 1 Introduction There has been a growing interest in direct policy-gradient methods for approximate planning in large Markov decision problems (MDPs). Unfortunately, the standard gradient descent rule is noncovariant. In this paper, we present a covariant gradient by defining a metric based on the underlying structure of the policy.
Neural Implementation of Bayesian Inference in Population Codes
Such a coding strategy is called population coding. It is conceivable that population coding has advantage of being robust to the fluctuation in a single neuron's activity. However, people argue that population coding may have other computationally desirable properties. One such property is to provide a framework for encoding complex objects by using basis functions [1]. This is inspired by the recent progresses in nonlinear function approximation, such as, sparse coding, overcomplete representationand kernel regression. These methods are efficient and show some interesting neuron-like behaviors [2,3].
Modularity in the motor system: decomposition of muscle patterns as combinations of time-varying synergies
The question of whether the nervous system produces movement through the combination of a few discrete elements has long been central to the study of motor control. Muscle synergies, i.e. coordinated patterns of muscle activity, have been proposed as possible building blocks. Here we propose a model based on combinations of muscle synergies with a specific amplitudeand temporal structure. Time-varying synergies provide a realistic basis for the decomposition of the complex patterns observed in natural behaviors. To extract time-varying synergies from simultaneous recordingof EMG activity we developed an algorithm which extends existing nonnegative matrix factorization techniques.