Sloan (1988), Cohen (1997), and was most recently highlighted in Avrim Blum's In this work, we focus on learning halfspaces with Massart noise [MN06]: Definition 1.1 A learning algorithm is given i.i.d. The question is whether a polynomial time algorithm exists.

Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and the stochastic gradient thermostat. While finite-time convergence properties of the SGLD with a 1st-order Euler integrator have recently been studied, corresponding theory for general SG-MCMCs has not been explored. In this paper we consider general SG-MCMCs with high-order integrators, and develop theory to analyze finite-time convergence properties and their asymptotic invariant measures. Our theoretical results show faster convergence rates and more accurate invariant measures for SG-MCMCs with higher-order integrators.

Neural Information Processing Systems http://nips.cc/

We consider the task of sampling with respect to a log concave probability distribution. The potential of the target distribution is assumed to be composite, i.e., written as the sum of a smooth convex term, and a nonsmooth convex term possibly taking

This estimator has been the subject of extensive empirical inquiry [e.g.

Neural Information Processing Systems http://nips.cc/

We consider Empirical Risk Minimization (ERM) in the context of stochastic optimization with exp-concave and smooth losses--a general optimization framework that captures several important learning problems including linear and logistic regression, learning SVMs with the squared hinge-loss, portfolio selection and more. In this setting, we establish the first evidence that ERM is able to attain fast generalization rates, and show that the expected loss of the ERM solution in d dimensions converges to the optimal expected loss in a rate of d/n. This rate matches existing lower bounds up to constants and improves by a log n factor upon the state-of-the-art, which is only known to be attained by an online-to-batch conversion of computationally expensive online algorithms.

The Jaccard index is a standard statistics for comparing the pairwise similarity between data samples. This paper investigates the problem of e stimating a Jaccard index matrix when there are missing observations in data sam ples. Starting from a Jaccard index matrix approximated from the incomplete dat a, our method calibrates the matrix to meet the requirement of positive semi-d efiniteness and other constraints, through a simple alternating projection algo rithm. Compared with conventional approaches that estimate the similarity matr ix based on the imputed data, our method has a strong advantage in that the calibrate d matrix is guaranteed to be closer to the unknown ground truth in the Frobenius norm than the un-calibrated matrix (except in special cases they are iden tical). We carried out a series of empirical experiments and the results confirmed ou r theoretical justification. The evaluation also reported significantly improved r esults in real learning tasks on benchmark datasets.

This paper addresses the question of which of these tasks are asymptotically solved by sorting by decreasing order of expected utility, for some suitable notion of utility, or, equivalently, when is square loss regression consistent for ranking via score-and-sort?