Neural Information Processing Systems http://nips.cc/

Over the past years data adaptiveness has proven to be crucial to the success of learning algorithms. The objective function underlying "big data" applications often demonstrates intricate structure: the scale and smoothness are often unknown and may change substantially in between different regions/directions, [

The complexity of the traditional LP solver is still at least quadratic in the problem dimension, i.e., the

Neural Information Processing Systems http://nips.cc/

Maximum likelihood for LDS estimation, on the other hand, has several advantages.

The submodular function minimization (SFM) problem arises in problems in image segmentation or MAP inference tasks in Markov Random Fields. Landmark results in combinatorial optimization give polynomial-time exact algorithms for SFM. However, the high-degree polynomial dependence in the running time is prohibitive for large-scale problem instances. The main objective in this context is to develop fast and scalable SFM algorithms.

Among them Nesterov's accelerated gradient algorithm [Nesterov, 1983] is proven to be optimal on the

The spikes highlighted in red are part of a repeating motif.

The goal of the player is to cumulate as much reward as possible over time.

We consider the parametric learning problem, where the objective of the learner is determined by a parametric loss function. Employing empirical risk minimization with possibly regularization, the inferred parameter vector will be biased toward the training samples. Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the out-of-sample performance. A classical cross validation strategy is the leave-one-out cross validation (LOOCV) where one sample is left out for validation and training is done on the rest of the samples that are presented to the learner, and this process is repeated on all of the samples. LOOCV is rarely used in practice due to the high computational complexity. In this paper, we first develop a computationally efficient approximate LOOCV (ALOOCV) and provide theoretical guarantees for its performance. Then we use ALOOCV to provide an optimization algorithm for finding the regularizer in the empirical risk minimization framework. In our numerical experiments, we illustrate the accuracy and efficiency of ALOOCV as well as our proposed framework for the optimization of the regularizer.