As a non theory person I was able to follow the motivation, the problem set-up and the main result. The intuition that not all queries Q(j,k) needs to be issued conditioned on the past queries due to the fact that we're in a metric space. The bounds looks fine, mostly it is a construction of the confidence in the estimation of Q(j,k), again, under the constraint that Q(j,k) is a metric space with metric-ish properties. As a piece of technical achievement this paper is just fine. But it can improve in the following sense of story-telling and organisation: 1) we're doing an optimal query problem, namely, querying a noisey oracle Q(j,k) to construct a nearest-neighbor graph G(x_i,x_j).

This paper defines a new learning problem of learning 1-NN graph on a metric space consisting of n points using a noisy distance oracle. Ignoring the metric structure, the problem can be viewed n separate best arm identification problems (for each node find its nearest neighbor by calling the oracle). The best arm identification problem can solved by UCB-like methods that maintain confidence intervals for each distance and has been studied extensively in the literature. The paper proposes an improved algorithm that exploits the triangle inequality and symmetry of the underlying metric in order to improve the confidence intervals. As the paper shows that the improvement doesn't help much for the worst-case metric space.

The paper addresses the problem of predicting the outcome of an action chosen from a set of possible actions with Distributionally Robust Nearest-Neighbor Regression. Additionally to the description of the method and its theoretical analysis, an application to finding optimal prescriptions for patients with hypertension is studied. The reviewers found that the paper was written in a clear manner. The ideas of the paper were found interesting and novel. The work brings a non trivial theoretical analysis.

Additional Feedback: I read all the reviews and the rebuttal. I agree with the authors that the proposed method is different from learned pseudo-exemplars in the embedding space as in VampVAE, and this work uses real exemplars in the image space. However, I am not convinced that randomly sampling exemplars in the data space with some heuristics based on LOO and trivial exemplar subsampling as regularizations on toy datasets is a significant contribution extending the exemplar-based prior in VampVAE. A possible limitation of the proposed Exemplar VAE is that, the generative model might not learn much beyond reconstruction, instead, it only produces some random samples that stay close to epsilon-ball of training data points. It's possible that Exemplar VAE even performs no better than a deterministic autoencoder with tiny Gaussian noise added to latent codes and k-means regularization in the latent space. VampVAE doesn't have this issue.

And since the ideas are rather incremental Meta-reviewer recommendations: The paper is borderline. R2 is considering increasing the score. R4 recommends rejection based on the lack of novelty compared to the VampPrior and that the paper conducts small-scale non-challenging experiments that doesn't require approximate nearest-neighbor search. He proposes to run the method on ImageNet but I believe this cannot be a condition for acceptance since not everyone has the potential for running such experiments. Furthermore, the paper already covers quite a lot in experiments with MNSIT, Omniglot, FashionMNIST and CelebA, and classification. I believe that R4's concern for novelty are successfully addressed in the rebuttal.

Large-scale approximate nearest neighbor search (ANN) has been gaining attention along with the latest machine learning researches employing ANNs. If the data is too large to fit in memory, it is necessary to search for the most similar vectors to a given query vector from the data stored in storage devices, not from that in memory. The storage device such as NAND flash memory has larger capacity than the memory device such as DRAM, but they also have larger latency to read data. Therefore, ANN methods for storage require completely different approaches from conventional in-memory ANN methods. Since the approximation that the time required for search is determined only by the amount of data fetched from storage holds under reasonable assumptions, our goal is to minimize it while maximizing recall. For partitioning-based ANNs, vectors are partitioned into clusters in the index building phase. In the search phase, some of the clusters are chosen, the vectors in the chosen clusters are fetched from storage, and the nearest vector is retrieved from the fetched vectors. Thus, the key point is to accurately select the clusters containing the ground truth nearest neighbor vectors. We accomplish this by proposing a method to predict the correct clusters by means of a neural network that is gradually refined by alternating supervised learning and duplicated cluster assignment. Compared to state-of-the-art SPANN and an exhaustive method using k-means clustering and linear search, the proposed method achieves 90% recall on SIFT1M with 80% and 58% less data fetched from storage, respectively.

Prior case retrieval (PCR) is crucial for legal practitioners to find relevant precedent cases given the facts of a query case. Existing approaches often overlook the underlying semantic intent in determining relevance with respect to the query case. In this work, we propose LeCoPCR, a novel approach that explicitly generate intents in the form of legal concepts from a given query case facts and then augments the query with these concepts to enhance models understanding of semantic intent that dictates relavance. To overcome the unavailability of annotated legal concepts, we employ a weak supervision approach to extract key legal concepts from the reasoning section using Determinantal Point Process (DPP) to balance quality and diversity. Experimental results on the ECtHR-PCR dataset demonstrate the effectiveness of leveraging legal concepts and DPP-based key concept extraction.

The writing could be improved, but it's in general understandable. However, citation quality can be improved. In particular, it seems to me that NSG and HNSW are actually using the same pruning rule (which results in approximate relative neighborhood graph). I really like your updated version, which reduces the number hops (and I haven't seen this pruning variant before)! Detailed comments: Abstract and further: base points sounds like a strange term, do you mean domain points? Please, find a more specific-generic citation that describes this phenomena.

In post rebuttal discussions, reviewers concurred in subsequent discussions that the paper presents solid state of art implementation and very impressive results, which will have good impact for practitioners. This significant impact by itself was worthy of publication.

Weaknesses: Unfortunately, I strongly believe that this paper will have a very limited attraction from the research community since derivations are done for binary classification and the nearest neighbor classification is no longer popular as before as there are numerous good alternatives. To validate my claim, I looked at the recent Neurips 2019 paper cited as [45] which is quite similar to this paper. In one year, it is cited only once. This is quite natural in my opinion since deep neural networks dominated classification and there are good alternatives to the nearest neighbor classification for large-scale data as hashing, approximate nearest neighbor classification methods, etc. Especially, unsupervised and supervised hashing methods are quite popular for large-scale data with high-dimensional feature spaces. Therefore, I strongly believe that the impact of the paper is very limited and it will attract a very few attention from research community.