The optimal first-order algorithm of Nesterov has linear convergence for such problem but the constant depends on the square root of the condition number k. The authors consider the situation where one has access to the expensive full gradient of the objective as well as a cheap stochastic gradient oracle. They propose a hybrid algorithm which only requires O(log 1/eps) calls to the full gradient oracle (independent of the condition number) and O(k^2 log(1/eps)) calls to the cheaper stochastic gradient oracle -- as long as the condition number is not too big, this could be faster in theory. The main idea behind their algorithm(called Epoch Mixed Gradient Descent - EMGD) is to replace a full gradient step (called an epoch) with a fixed number O(k^2) of mixed gradient steps which use a combination of the full gradient (computed once for the epoch) and stochastic gradients (which vary within an epoch). By taking the average of the O(k^2) iterates within an epoch, they can show a constant decrease of the suboptimality *independent* of the condition number, which is why the number of required full gradient step computations (the number of epochs) is independent from the condition number. They provide a simple and complete self-contained proof of their convergence rate, but no experiment.

It is important to also demonstrate the robustness of the proposed algorithm on data relative to the naive model.

As explained in Section 2.3, we use the linear metric elicitation procedure [ Let the oracle's scale invariant metric be The algorithm is summarized in Algorithm 2. The algorithm also uses the following Parameterizing the boundary of the enclosed sphere S . Suppose that the oracle's linear metric is The number of cycles in coordinate-wise search is fixed to four. Please do not confuse it with the sensitive group index. 's are vertices: Any supporting hyperplane with slope's are vertices of the convex set. The following discussion is extended from [21] to multiple groups setting and provided here for completeness.

What is a fair performance metric?

However, both steps suffer from high computational and memory complexity, preventing the application of SSC to large-scale datasets.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The authors present a hierarchical extension of the IRM for network modelling using the key ideas from the Bayesian rose tree paper: 1) that the hierarchy is used to specify a mixture over consistent partitions of the nodes 2) that this hierarchy can be learnt using an efficient greedy agglomerative procedure. Qualitative results on the Sampson's monks dataset, and full NIPS dataset, and quantitative results on the NIPS-234 dataset are presented. The proposed inference is computational much cheaper than the IRM, whilst obtaining similar predictive performance. The paper is very well written and the exposition of the key ideas is clear.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. This paper proposes an approach to stochastic multi-objective optimization. The main idea is simply described: optimize a single objective while taking other objectives as constraints. The authors proposes a primal-dual stochastic optimization algorithm to solve the problem and prove that it achieves (for the primal objective) the optimal 1/\sqrt{T} convergence rate. As far as I am concerned, the theory is solid and it does provide a good insight into the problem of interest.

Classical learning theory suggests that the optimal generalization performance of a machine learning model should occur at an intermediate model complexity, with simpler models exhibiting high bias and more complex models exhibiting high variance of the predictive function.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The paper studies the collision probability of the following hashing scheme for points in R^D: the hash function picks a random vector of D i.i.d. The same scheme for 2-stable distribution has been studied before (known as sim-hash). The main result shows that the collision probability for alpha=1 on binary data can be approximated by a function of the chi-square similarity. The bound for general data is pointed out by the authors themselves to be far from the true collision probability so it is not clear what it means, especially regarding the comparison between that bound and the chi square similarity. The paper provides some experiments to show that the collision probability is approximately equal to some functions of the chi square similarity, and thus one can approximate chi square similarity from the collision probability and these functions. However, in this process, I think we are no longer able to use linear SVM or efficient near neighbor search (advantages 2 and 3 in the introduction) and have to use kernel SVM and exhaustive search instead. When using linear SVM, we use the kernel implicitly defined by the LSH and there is inadequate explanation of how useful it is (chi square is useful but it does not mean any function of it is also useful).