This paper seems to accomplish two feats at once: it provides a rather deep dive into the specific topic of gradient flows w.r.t. MMD, while it also lays out some original propositions and theorems that establish the paper's main contributions. The first two sections of the paper are excellent and provide a solid introduction to the material the subsequent sections. Per C1, it appears this is fully realized in Proposition 7 in Section 3.2. As an outsider to this level of detail in the field, it is unclear how strigent this assumption is to provide convergence to a global optimum.

Summary and Contributions: This paper focuses on the problem of clustering in high dimension. K-means clustering is an extremely popular tool (especially in biomedical applications). However, as underlined by the authors, its performance is severely hindered in high-dimensional space --- leaving the data analyst no chance but to (a) apply some dimensionality reduction technique before performing the clustering or (b) selecting the features that are the most informative for the clustering and apply k-means on a subset of the features. This paper proposes a version of the later approach, choosing a sparse and interpretable subset of features. The setting is the following.

The reviewers appreciate the algorithmic contributions of this paper and believe it will be on interest to the community.

Weaknesses: The main issue I have with the paper is about the novelty of the results. The authors mention that previous work on linear regression is not as general as current work. In particular, they either only allow isotropic features or signal. This paper which is arXived about a month before the NeurIPS deadline seems to do both: [1] Emami, Melikasadat, et al. "Generalization error of generalized linear models in high dimensions." The results of this paper allow to characterize the exact generalization error in the same asymptotic limit for Guassian data with general covariance and any regularization, which includes the \ell_2 type regularzations considered here, as well as more general regularizations like general \ell_p norms. Here are my understanding of the differences of the results of the two papers: - In [1] the authors allow for a Gaussian feature with any covariance matrix, whereas your paper allow non-Gaussina features so long as they have bounded 12th centered-moment.

The paper received three positive reviews. Most of the minor concerns raised in the initial reviews have been addressed in the rebuttal. The area chair agrees with the reviewers' assessment and follows their recommendation.

The paper tackles the problem of predicting the outcome of an action chosen from a set of possible actions, The outcome is a function of the action, having a linear component, non-linear component and some additive noise. The idea is first finding a linear function minimizing the deviation from the outcomes, for every distribution which is "close" to the empirical distribution (by the Wasserstein distance). Idea which was analyzed before. The idea added in the paper is using the resulting linear-regression coefficient to build a metric upon samples from the same group and then produce prediction which is the average of the outcomes for the K-nearest neighbors. This way the prediction can leverage not only the private history of the specific instance but also the outcomes of "close" instances.

The SMILE algorithm is basically the Nadaraya-Watson estimator (with the Gaussian kernel with the Mahalanobis metric instead of the Euclidean metric) where the support vectors are also learnt instead of using training points as support vectors. It is not clear how much advantage does the SMILE classifier get by learning the representative instances. I suspect that they may account for a lot of the advantage SMILE has over other algorithms, since the SMILE classifier is the very simple Nadaraya-Watson estimator, as pointed out above. Moreover, the other competitor algorithms were not offered a chance to similarly learn nice prototypes. One way to rebut this criticism would be to run SMILE but restrict it to using a subset of training points or else award all other methods e.g. It would have been nice if some contemporary applications with VAE or deep metric learning could have been explored.

The four referees support acceptance for the contribution and I also recommend acceptance. However, please consider revising your paper in order to address the lack of novelty of Lemma 1, to add the generalization bound of SMILE and to precise the fact that your result is rather general and not purely specific to metric learning.

Additional Feedback: - Two-cluster case is a convex optimization of the linear model and has been investigated in a bit different context [21]. Therefore, the three cluster case is more untrivial and exciting. However, I am not sure that the DMFT formulation in the three-cluster case is tractable enough to analyze SGD dynamics' behavior. Since the three-cluster case is non-convex optimization, I suspect that DMFT equations (20) have some local optima. If this is the case, it becomes unclear how typical the dynamics shown in experiments on three-cluster cases are.

The reviewers agree that the techniques leveraged in the paper should be of interest to the wider NeurIPS community. Furthermore, even though the setting analyzed is relatively simple, the analysis is challenging, and understanding the effects of batch size is a problem of broad interest.