Neural Information Processing Systems http://nips.cc/

It is defined w.r.t. a finite number of finite sets, all of the same cardinality. A feasible solution consists in as many bijections as there are pairs of distinct sets. These bijections are constrained to be consistent in the following sense: For any three sets, A, B, C, if a in A is mapped to b in B and b in B is mapped to c in C, then a needs to be mapped to c. The objective function is defined w.r.t.

Meta-learning, or 'learning to learn' [ However, it can be any useful knowledge which can be learned in the meta-training stage. This setting is intuitive and theoretically sound.

The optimal first-order algorithm of Nesterov has linear convergence for such problem but the constant depends on the square root of the condition number k. The authors consider the situation where one has access to the expensive full gradient of the objective as well as a cheap stochastic gradient oracle. They propose a hybrid algorithm which only requires O(log 1/eps) calls to the full gradient oracle (independent of the condition number) and O(k^2 log(1/eps)) calls to the cheaper stochastic gradient oracle -- as long as the condition number is not too big, this could be faster in theory. The main idea behind their algorithm(called Epoch Mixed Gradient Descent - EMGD) is to replace a full gradient step (called an epoch) with a fixed number O(k^2) of mixed gradient steps which use a combination of the full gradient (computed once for the epoch) and stochastic gradients (which vary within an epoch). By taking the average of the O(k^2) iterates within an epoch, they can show a constant decrease of the suboptimality *independent* of the condition number, which is why the number of required full gradient step computations (the number of epochs) is independent from the condition number. They provide a simple and complete self-contained proof of their convergence rate, but no experiment.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. This paper proposes an approach to stochastic multi-objective optimization. The main idea is simply described: optimize a single objective while taking other objectives as constraints. The authors proposes a primal-dual stochastic optimization algorithm to solve the problem and prove that it achieves (for the primal objective) the optimal 1/\sqrt{T} convergence rate. As far as I am concerned, the theory is solid and it does provide a good insight into the problem of interest.

Due to state transitions, it is challenging to balance the contribution from each dimension for achieving near-optimality.

We consider the problem of fitting variational posterior approximations using stochastic optimization methods.

We consider the problem of fitting variational posterior approximations using stochastic optimization methods.

We study online convex optimization in a setting where the learner seeks to minimize the sum of a per-round hitting cost and a movement cost which is incurred when changing decisions between rounds.