While my concerns were given significant attention in the rebuttal, I feel they were not fully addressed. In particular, regarding comparison with deep ADMM-net and LDAMP, the authors argue that these methods need more training data/training time. However, training time is normally not a big issue (you only train your model once, does it matter if it takes 2 hours or 10?). The *size* of the training data is however important, but no experiments are provided to show superior performance of the proposed method with respect to the the size of training data. This is surprising given that in l. 62. the authors say they use "much less training data" (addressing the challenge of "scarcity of training data" mentioned in l.4 in abstract), without referring back to this claimed contribution anywhere in the paper!

This paper that is built on top of Bhaskara etal. Moreover, the described analysis is related to the recovery of assemblies of neurons. Overall, the paper gave a detailed analysis of the tensor decomposition using l 1and a general case although a better differentiation with respect to [3] is needed. Moreover, the proposed application of recovering assemblies of neurons is misleading. For instance, the application aim is not clear enough because the abstract mentioned recovering assemblies of neurons but on the main text and problem formulation aims to talks about assembly association and the structure intersection of cell assembly intersection, and also how the problem is posed since there is ambiguous definitions, e.g. Moreover, what would it happen if there is no association among neurons but there is still an overlap as have been shown in different papers [1,2]?

Update: I keep my initial rating. As a potential improvement for the paper, I see the authors only tackle the non strictly convex case. I am curious how the result would be modified if the authors assumed strong convexity. Original review: In this paper the authors introduce SVRG OL, a distributed stochastic optimization method for convex optimization. Inspired by SVRG, the authors first compute a high precision estimate of a gradient at an anchor point v using a large number of samples.

This paper provides theoretical analysis and empirical examples for two phenomenon in active learning. The first is it could be possible that the 0-1 loss on subset of the entire dataset generated uncertainty sampling is smaller than learning using the whole dataset. The second is uncertainty sampling could "converge" to different models and predictive results. In the analysis, it is shown that the reason for these is the expected gradient of the "surrogate" loss of the most uncertain point is in the direction of the gradient of the current 0-1 loss. This result is based on the setup that the most uncertain point is sampled from a minipool that is a subset sampled without replacement randomly from the entire set.

This paper studies the multi-response regression model, and in particular, the alternating minimization algorithm for the problem . In contrast to prior work, this paper makes the following improvements: 1. It does not require the resampling assumptions that usually abound in showing results on alternating procedures. With good initialization, the procedure without resampling is able to achieve what is usually the minimax rate of such problems. The major technique is that of generic chaining, which allows the authors to prove bounds that hold uniformly over all iterates. I liked the paper and the result overall seems quite interesting, in particular points 1 and 2 above.

This paper presents a stochastic gradient Monte Carlo approach defined on a Cartesian product of SE(3), a domain commonly used to characterize problems in structure-from-motion (SFM) among other areas. The algorithm is parameterized by an inverse temperature such that when the value goes to inifinity, the algorithm is implicitly operating on a delta function with it's peak at the maximum of the base distribution. The proposed algorithm is formulated as a SDE and a splitting scheme is proposed to integrate it. A theoretical analysis on the SDE and its discretization is explored, showing that 1) the resulting Markov process has the appropriate invariant distribution and 2) the sampler will draw samples close to the maximum of the posterior (in terms of expectation of the unnormalized log posterior). Along with the algorithm, a model is defined using the Bingham distribution to characterize typical SFM posteriors which is then used to perform experiments with the algorithm.

SUMMARY: This paper proposes a method to incorporate prior knowledge into the spectral learning algorithm for predictive state representations (PSR). The prior knowledge consists of an imperfect/incomplete state representation which is'refined' and'completed' by the learning algorithm. This contribution addresses one of the main caveats of spectral methods: while these methods are fast and consistent, they tend to perform worse than local methods (e.g. By leveraging domain specific knowledge, the proposed algorithm overcomes this issue. The proposed extension, PSR-f, is relatively straightforward: the belief vector at each time step is the concatenation of the user-specified state representation f with a learned state representation b; the parameters of b are learned in the same fashion as for the classical method by solving linear regression problems constrained to the row space of the concatenation of some Hankel/system matrices (e.g. now mapping [P(T h); P(h)f(h)] to P(oT h) for each B_o).

I think the rebuttal is prepared very well. Although the assumption of a single component approximating the local decision boundary is quite strong, the paper nonetheless offers a good, systematic approach to interpreting black box ML systems. It is an important topic and I don't see a lot of studies in this area. Overview In an effort to improve scrutability (ability to extract generalizable insight) and explainability of a black box target learning algorithm the current paper proposes to use infinite Dirichlet mixture models with multiple elastic nets (DMM-MEN) to map the inputs to the predicted outputs. Any target model can be approximated by a non-parametric Bayesian regression mixture model.

Summary The paper describes a generic 2nd order stochastic optimisation scheme exploiting curvature information to improve the trade-off between convergence speed und computational effort. It proposes an extension to the approximate natural gradient method KFAC where the Fisher information matrix is restricted to be of Kronecker structure. The authors propose to relax the Kronecker constraint and suggest to use a general diagonal scaling matrix rather than a diagonal Kronecker scaling matrix. This diagonal scaling matrix is estimated from gradients along with the Kronecker eigenbasis. Quality The idea in the paper is convincing and makes sense.

The authors analyze the use of random Fourier Features (RFF) for training linear support vector machines in the RFF feature space. Their result bounds the expected risk difference of the hinge loss (i.e. the generalization error on unseen data) for a number of different scenarios. The analysis is based on a number of assumptions, on the noise distribution, on the RKHS containing the optimal classifier, and most crucially on access to the optimal feature weights by Bach 2017. The first main result is a fast rate for kernels whose spectrum decays polynomially when the Bayes classifier is in the feature space. The second result is a fast rate specifically for the Gaussian kernel when the Bayes classifier is not necessarily in the feature space, but the data is separable by some minimum distance.