Industry
Stochastic Mixed-Signal VLSI Architecture for High-Dimensional Kernel Machines
Genov, Roman, Cauwenberghs, Gert
A mixed-signal paradigm is presented for high-resolution parallel innerproduct computationin very high dimensions, suitable for efficient implementation ofkernels in image processing. At the core of the externally digital architecture is a high-density, low-power analog array performing binary-binary partial matrix-vector multiplication. Full digital resolution is maintained even with low-resolution analog-to-digital conversion, owing torandom statistics in the analog summation of binary products. A random modulation scheme produces near-Bernoulli statistics even for highly correlated inputs. The approach is validated with real image data, and with experimental results from a CID/DRAM analog array prototype in 0.5
Minimax Probability Machine
Lanckriet, Gert, Ghaoui, Laurent E., Bhattacharyya, Chiranjib, Jordan, Michael I.
When constructing a classifier, the probability of correct classification offuture data points should be maximized. In the current paper this desideratum is translated in a very direct way into an optimization problem, which is solved using methods from convex optimization.We also show how to exploit Mercer kernels in this setting to obtain nonlinear decision boundaries. A worst-case bound on the probability of misclassification of future data is obtained explicitly. 1 Introduction Consider the problem of choosing a linear discriminant by minimizing the probabilities thatdata vectors fall on the wrong side of the boundary. One way to attempt to achieve this is via a generative approach in which one makes distributional assumptions aboutthe class-conditional densities and thereby estimates and controls the relevant probabilities. The need to make distributional assumptions, however, casts doubt on the generality and validity of such an approach, and in discriminative solutionsto classification problems it is common to attempt to dispense with class-conditional densities entirely.
Probabilistic Inference of Hand Motion from Neural Activity in Motor Cortex
Gao, Yun, Black, Michael J., Bienenstock, Elie, Shoham, Shy, Donoghue, John P.
Statistical learning and probabilistic inference techniques are used to infer thehand position of a subject from multi-electrode recordings of neural activityin motor cortex. First, an array of electrodes provides training dataof neural firing conditioned on hand kinematics. We learn a nonparametric representationof this firing activity using a Bayesian model and rigorously compare it with previous models using cross-validation. Second, we infer a posterior probability distribution over hand motion conditioned on a sequence of neural test data using Bayesian inference. The learned firing models of multiple cells are used to define a non-Gaussian likelihood term which is combined with a prior probability for the kinematics. A particle filtering method is used to represent, update, and propagate the posterior distribution over time. The approach is compared withtraditional linear filtering methods; the results suggest that it may be appropriate for neural prosthetic applications.
Generalization Performance of Some Learning Problems in Hilbert Functional Spaces
We investigate the generalization performance of some learning problems inHilbert functional Spaces. We introduce a notion of convergence of the estimated functional predictor to the best underlying predictor, and obtain an estimate on the rate of the convergence. This estimate allows us to derive generalization bounds on some learning formulations.
Orientational and Geometric Determinants of Place and Head-direction
The model can predict the response ofindividual cells and populations to parametric manipulations of both geometric (e.g.O'Keefe & Burgess, 1996) and orientational (Fenton et aI., 2000a) cues, extending a previous geometric model (Hartley et al., 2000). It provides a functional description of how these cells' spatial responses are derived from the rat's environment and makes easily testable quantitative predictions. Consideration ofthe phenomenon of remapping (Muller & Kubie, 1987; Bostock et aI., 1991) indicates that the model may also be consistent with nonparametric changesin firing, and provides constraints for its future development.
Effective Size of Receptive Fields of Inferior Temporal Visual Cortex Neurons in Natural Scenes
Trappenberg, Thomas P., Rolls, Edmund T., Stringer, Simon M.
Inferior temporal cortex (IT) neurons have large receptive fields when a single effective object stimulus is shown against a blank background, but have much smaller receptive fields when the object is placed in a natural scene. Thus, translation invariant object recognition is reduced in natural scenes, and this may help object selection. We describe a model which accounts for this by competition within an attractor in which the neurons are tuned to different objects in the scene, and the fovea has a higher cortical magnification factor than the peripheral visual field. Furthermore, weshow that top-down object bias can increase the receptive field size, facilitating object search in complex visual scenes, and providing a model of object-based attention. The model leads to the prediction that introduction of a second object into a scene with blank background will reduce the receptive field size to values that depend on the closeness of the second object to the target stimulus. We suggest that mechanisms of this type enable the output of IT to be primarily about one object, so that the areas that receive from IT can select the object as a potential target for action.
A theory of neural integration in the head-direction system
Hahnloser, Richard, Xie, Xiaohui, Seung, H. S.
Integration in the head-direction system is a computation by which horizontal angularhead velocity signals from the vestibular nuclei are integrated toyield a neural representation of head direction. In the thalamus, the postsubiculum and the mammillary nuclei, the head-direction representation has the form of a place code: neurons have a preferred head direction in which their firing is maximal [Blair and Sharp, 1995, Blair et al., 1998,?]. Integration is a difficult computation, given that head-velocities can vary over a large range. Previous models of the head-direction system relied on the assumption that the integration is achieved in a firing-rate-based attractor network with a ring structure. In order to correctly integrate head-velocity signals during high-speed head rotations, very fast synaptic dynamics had to be assumed. Here we address the question whether integration in the head-direction system is possible with slow synapses, for example excitatory NMDA and inhibitory GABA(B) type synapses. For neural networks with such slow synapses, rate-based dynamics are a good approximation of spiking neurons[Ermentrout, 1994]. We find that correct integration during high-speed head rotations imposes strong constraints on possible network architectures.
3 state neurons for contextual processing
Kepecs, รdรกm, Raghavachari, S.
Neurons receive excitatory inputs via both fast AMPA and slow NMDA type receptors. We find that neurons receiving input via NMDA receptors can have two stable membrane states which are input dependent. Action potentials can only be initiated from the higher voltage state. Similar observations have been made in several brainareas which might be explained by our model. The interactions betweenthe two kinds of inputs lead us to suggest that some neurons may operate in 3 states: disabled, enabled and firing. Suchenabled, but non-firing modes can be used to introduce context-dependent processing in neural networks. We provide a simple example and discuss possible implications for neuronal processing andresponse variability. 1 Introduction Excitatory interactions between neurons are mediated by two classes of synapses: AMPA and NMDA.
ACh, Uncertainty, and Cortical Inference
Acetylcholine (ACh) has been implicated in a wide variety of tasks involving attentional processes and plasticity. Following extensive animal studies, it has previously been suggested that ACh reports on uncertainty and controls hippocampal, cortical and cortico-amygdalar plasticity. We extend this view and consider its effects on cortical representational inference, arguing that ACh controls the balance between bottom-up inference, influenced by input stimuli, and top-down inference, influenced by contextual information. We illustrate our proposal using a hierarchical hidden Markovmodel.