I think the clarity of your presentation can be strengthened. Sure, I'd like to integrate out the hyperparameters (and thus average over a class of models), but if I can't do that, then I'm happy to learn them and select a single model. Indeed, I'd question your claim that the Bayesian perspective is one of Bayesian model selection---you're just moving one more level up the hierarchy (hyper-hyper parameters!), but wouldn't it be even better if we could integrate over different models M? This might actually be possible within your framework---you're using Bayesian optimization to maximize a function, but the recent probabilistic numerics work on Bayesian quadrature suggests you could instead use BO to do integration! I think run an optimization to approximate the model evidence, is that right? If so, it's confusing me because the model evidence approximation isn't discussed until later -- page 5 while the acquisition function is discussed on page 3. 6) The priors you suggest are reasonable, though I wonder if it'd be better to use a distribution like Student t (see some work by Vehtari and colleagues) in place of a Normal.

The authors develop an very interesting method to the hard problem of computing the partition function of GM. In particular, there is no method to compute Z for general graphs accurately and log(Z) is a particularly interesting component in many fields: it is related to the likelihood in Machine Learning, it gives all the statistical information of a system in statistical mechanic, ... I think therefore that any effort toward that direction is an important contribution. Concerning the work presented here, I have some questions and concerns that I would like the authors to answer: 1) the authors mentioned that BP in general fail to converge on general graph but that more involved convergent alternative exists. I think the author should be more specific, precising if all the convergent algorithms do provide the same estimation of the BP partition function and it they are indeed linear in the system size. The 2-reg loop can be proven to converge in polynomial time but the full loop does not have these garranty.

The main contribution in this work is showing that in the context of matrix completion, when equipped with a good initialization, the sequence of the solutions produced by a widely used (but without theoretical guarantee) SGD algorithm converges linearly to the true matrix. The paper reads well and most of theoretical result looks sound. But I have some concerns as follows. Is it possible to access fewer samples, e.g., "O(mu * d * k * log(d))" while still ensuring global convergence? Theorem 3.1 says "for any fixed T 1, with probability at least 1 - T/(d 10), we have linear convergence". That means, if we run the algorithm (i.e., repeatedly reveal the samples) for a longer time, we have a LOWER confidence to obtain the true matrix.

Strength: --The paper provides the theoretical analysis of approximation guarantees and a generalization bound for the class of tensor-valued regression functions. Weakness: --A major drawback is that the novelty and contribution is rather limited. The key idea and the model of this paper is actually equivalent to the HOPLS in the following paper: [Zhao et. In HOPLS, it assumes the tensor input has low-rank structure and also the tensor output has low-rank structure, and the link of them is established in the common latent space. And then follows a regression step against the projected latent variables.

I find the result interesting and it is a nice addition to the literature but I do not find it very surprising in itself. Intuitively, D-sampling algorithms do not need to know k in advance, thus, if the algorithm is efficient for finding k centers (which we know it is from [4]), allowing it to pick beta k centers "has to" make the cost decrease significantly (as if the algorithm was solving an instance of the problem for beta k centers). Moreover, the result was known in the constant probability case (rather than expectation). One of the main issue of this paper is the lack of comparison with previous work. There are at least 4 recent papers that tackle the k-means problem (and two of them tackle the bi-criteria version) and 3 of them are not cited (all on arxiv): 1-- A bi-criteria approximation algorithm for k Means.

There are some issues with the conditions imposed on distributions in this paper. In particular, regarding the local bound hypothesis: in the proof of Lemma 2 epsilon may depend on x, so p * will be smaller than stated. As an example, if p(x) goes to 0 as x a when x goes to 0, then p *(x) goes to 0 as x (a 1). AFTER REBUTTAL: 1) The rebuttal here is helpful, but there seem to still be some issues with the local lower bounds hypothesis. They claim that it is very mild in light of Lemma 2, but Lemma 2 doesn't apply very well to the situation of p x a. I think that they should at least include what they've written in the rebuttal.

The mathematical derivation in this work is novel and interesting, but the writing can be improved. The derivation uses Hoeffding's inequality for noise on labels, with a mild assumption on the maximum noise. The analysis using KKT condition is straightforward, and it results in an interesting strategy for finding weights. I want to support the acceptance of this paper, but I have several concerns: 1) According to the derivation, the upper bound of the risk (P1) is given as C\lambda (line 112). With some simple calculations using appropriate parameters, this upper bound does not give a small value.

Note that in minibatch inference methods, at each iteration, a small subset of the data is sampled from the full dataset and used to make an update; these methods take advantage of the redundancy in data to perform inexpensive updates. In this paper, coresets reduce the total dataset size by, in some sense, approximating the dataset with a smaller group of (weighted) examples. However, when coresets are used in existing inference algorithms (such as these minibatch algorithms), it seems to me that a very similar procedure will occur: a small subset of this approximate, weighted dataset will be drawn, and used to make an update. I am not convinced this would actually speed up inference (i.e. In a sense, I feel that the main thing happening here is that the data is approximated in a smaller/compressed fashion; I can see how this might help with data storage concerns, but I don't see a great justification for why it would appreciably speed inference over existing minibatch methods (especially considering a coreset must be constructed before inference can proceed, which adds additional inference time to this method). One way to demonstrate this would be with timing comparison plots that explicitly show that coresets yield faster inferences given large datasets when compared to minibatch methods---however, no direct experiments of this sort are given.

I find the approach rather interesting, especially the broad and general definition of the problem makes the approach applicable to a wide range of problems. However, I was surprised by the absence of any reference to the seminal Robbins/Munro paper and also to the recent developments in stochastic gradient descent based sampling (see below). The authors do local gradient descent updates of coordinate blocks by computing partial gradients and adding noise in each asynchronous step. I was wondering, how this relates to the "usual" stochastic gradient descent update, i.e., given that the locally computed partial gradient will be based on delayed (noisy) variable states, a sequence of these noisy partial gradients would converge to the true partial gradient as well. Further, recent SGD based sampling has shown that adding noise to the variable states obtained by noisy gradient updates (as the authors do as well) provides good samples of the distribution underlying the optimal variable setting also in a non-convex setting. That being said, the work handed in remains valid, but it would have been interesting to compare the proposed approach to well established stochastic gradient methods.

My evaluation of the paper is balanced. Pros: The writing of the paper is very clear. The algorithms are very natural and suspected to work well on large scale OT problems. The dual formulation is a good idea to handle a stochastic algorithm. Everything is ok with all that points.