Many distance learning algorithms have been developed in recent years. However, few of them consider the problem when the class labels of training data are noisy, and this may lead to serious performance deterioration. In this paper, we present a robust distance learning method in the presence of label noise, by extending a previous non-parametric discriminative distance learning algorithm, i.e., Neighbourhood Components Analysis (NCA). Particularly, we analyze the effect of label noise on the derivative of likelihood with respect to the transformation matrix, and propose to model the conditional probability of the true label of each point so as to reduce that effect. The model is then optimized within the EM framework, with additional regularization used to avoid overfitting. Our experiments on several UCI datasets and a real dataset with unknown noise patterns show that the proposed RNCA is more tolerant to class label noise compared to the original NCA method.

We propose a new framework for single-channel source separation that liesbetween the fully supervised and unsupervised setting. Instead of supervision,we provide input features for each source signal and use convex methods toestimate the correlations between these features and the unobserved signaldecomposition. Contextually supervised source separation is a natural fit fordomains with large amounts of data but no explicit supervision; our motivatingapplication is energy disaggregation of hourly smart meter data (the separationof whole-home power signals into different energy uses). Here contextualsupervision allows us to provide itemized energy usage for thousands homes, a taskpreviously impossible due to the need for specialized data collection hardware.On smaller datasets which include labels, we demonstrate that contextualsupervision improves significantly over a reasonable baseline and existingunsupervised methods for source separation. Finally, we analyze the case of$\ell_2$ loss theoretically and show that recovery of the signal componentsdepends only on cross-correlation between features for different signals, not oncorrelations between features for the same signal.

Patients with epilepsy can manifest short, sub-clinical epileptic "bursts" in addition to full-blown clinical seizures. We believe the relationship between these two classes of events---something not previously studied quantitatively---could yield important insights into the nature and intrinsic dynamics of seizures. A goal of our work is to parse these complex epileptic events into distinct dynamic regimes. A challenge posed by the intracranial EEG (iEEG) data we study is the fact that the number and placement of electrodes can vary between patients. We develop a Bayesian nonparametric Markov switching process that allows for (i) shared dynamic regimes between a variable number of channels, (ii) asynchronous regime-switching, and (iii) an unknown dictionary of dynamic regimes. We encode a sparse and changing set of dependencies between the channels using a Markov-switching Gaussian graphical model for the innovations process driving the channel dynamics and demonstrate the importance of this model in parsing and out-of-sample predictions of iEEG data. We show that our model produces intuitive state assignments that can help automate clinical analysis of seizures and enable the comparison of sub-clinical bursts and full clinical seizures.

Learning Markov blanket (MB) structures has proven useful in performing feature selection, learning Bayesian networks (BNs), and discovering causal relationships. We present a formula for efficiently determining the number of MB structures given a target variable and a set of other variables. As expected, the number of MB structures grows exponentially. However, we show quantitatively that there are many fewer MB structures that contain the target variable than there are BN structures that contain it. In particular, the ratio of BN structures to MB structures appears to increase exponentially in the number of variables.

We introduce a new sequential Monte Carlo algorithm we call the particle cascade. The particle cascade is an asynchronous, anytime alternative to traditional particle filtering algorithms. It uses no barrier synchronizations which leads to improved particle throughput and memory efficiency. It is an anytime algorithm in the sense that it can be run forever to emit an unbounded number of particles while keeping within a fixed memory budget. We prove that the particle cascade is an unbiased marginal likelihood estimator which means that it can be straightforwardly plugged into existing pseudomarginal methods.

Forward inference techniques such as sequential Monte Carlo and particle Markov chain Monte Carlo for probabilistic programming can be implemented in any programming language by creative use of standardized operating system functionality including processes, forking, mutexes, and shared memory. Exploiting this we have defined, developed, and tested a probabilistic programming language intermediate representation language we call probabilistic C, which itself can be compiled to machine code by standard compilers and linked to operating system libraries yielding an efficient, scalable, portable probabilistic programming compilation target. This opens up a new hardware and systems research path for optimizing probabilistic programming systems.

We develop a technique for generalising from data in which models are samplers represented as program text. We establish encouraging empirical results that suggest that Markov chain Monte Carlo probabilistic programming inference techniques coupled with higher-order probabilistic programming languages are now sufficiently powerful to enable successful inference of this kind in nontrivial domains. We also introduce a new notion of probabilistic program compilation and show how the same machinery might be used in the future to compile probabilistic programs for efficient reusable predictive inference.

One of the goals of probabilistic inference is to decide whether an empirically observed distribution is compatible with a candidate Bayesian network. However, Bayesian networks with hidden variables give rise to highly non-trivial constraints on the observed distribution. Here, we propose an information-theoretic approach, based on the insight that conditions on entropies of Bayesian networks take the form of simple linear inequalities. We describe an algorithm for deriving entropic tests for latent structures. The well-known conditional independence tests appear as a special case. While the approach applies for generic Bayesian networks, we presently adopt the causal view, and show the versatility of the framework by treating several relevant problems from that domain: detecting common ancestors, quantifying the strength of causal influence, and inferring the direction of causation from two-variable marginals.

We perform the first study of the tradeoff space of access methods and replication to support statistical analytics using first-order methods executed in the main memory of a Non-Uniform Memory Access (NUMA) machine. Statistical analytics systems differ from conventional SQL-analytics in the amount and types of memory incoherence they can tolerate. Our goal is to understand tradeoffs in accessing the data in row- or column-order and at what granularity one should share the model and data for a statistical task. We study this new tradeoff space, and discover there are tradeoffs between hardware and statistical efficiency. We argue that our tradeoff study may provide valuable information for designers of analytics engines: for each system we consider, our prototype engine can run at least one popular task at least 100x faster. We conduct our study across five architectures using popular models including SVMs, logistic regression, Gibbs sampling, and neural networks.

Each volume scan consists radial velocity data. For any given pulse volume, radial of a sequence of sweeps during which the antenna velocity tells us the component of target velocity in rotates 360 degrees around a vertical axis while the direction of the radar beam, and we have no additional keeping its elevation angle fixed (figure 2). The result information about the component orthogonal of each sweep is a set of raster data products summarizing to the radar beam. However, the overall pattern of the the radar signal returned from targets within sweep often provides clear evidence about the true discrete pulse volumes, which are the portions of the target velocities. In this example, targets to the northeast atmosphere sensed at a particular antenna position (NE) of the radar station have negative radial and range from the radar. The coordinates of each velocities (dark colors), which means they are pulse volume (r, ϕ, ρ) are measured in a three-dimensional approaching the radar, and targets to the southwest polar coordinate system: r is the distance in (SW) of the radar station have positive radial velocities meters from the antenna, ϕ is the azimuth, which is (light colors), which means they are departing the angle in the horizontal plane between the antenna direction and a fixed reference direction (typically the radar station. We can infer that the targets (in this degrees clockwise from due north), and ρ is the elevation case, predominantly migrating birds) are moving uniformly angle, which is the angle between the antenna in a SW direction, as shown in panel (c). The direction and its projection onto the horizontal spiral pattern in the velocity image is due to changes plane.