Statistical Learning
Supplementary Material for Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
In this section, we provide the details and hyperparameters for SwA V pretraining and transfer learning. A.1 Implementation details of SwA V training First, we provide a pseudo-code for SwA V training loop using two crops in Pytorch style: # C: prototypes (DxK) # model: convnet + projection head # temp: temperature for x in loader: # load a batch x with B samples x_t = t(x) # t is a random augmentation x_s = s(x) # s is a another random augmentation z = model(cat(x_t, x_s)) # embeddings: 2BxD scores = mm(z, C) # prototype scores: 2BxK scores_t = scores[:B] scores_s = scores[B:] # compute assignments with torch.no_grad(): The latter methods require sharing the feature matrix across all GPUs at every batch which might become a bottleneck when distributing across many GPUs. A.2 Data augmentation used in SwA V We obtain two different views from an image by performing crops of random sizes and aspect ratios. We use the 1% and 10% splits specified in the official code release of SimCLR.
Export Reviews, Discussions, Author Feedback and Meta-Reviews
First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The paper looks at the problem of combining clustering and outlier detection. It is very well written and easy to read. The authors reuse an earlier facilities location without outliers formulation by Charikar et' al and their main contribution is the solution to the problem formulation. The FLO problem was shown to be intractable by the authors of that paper and no approximation algorithm exists that is both i) scalable and ii) comes with guarantees.
1. Our contribution. The DRLR informed KNN builds on a previously proposed method using OLS to inform
KNN. Y et, our key, differentiating contributions lie in that: It is hard to define "optimal Different from the classical metric learning works (e.g., Gottlieb et al. Gottlieb et al. [2017] focuses on the computational aspect of solving the metric regression problem. They significantly improve the computational efficiency compared to solving a convex program. A direct comparison of the two papers easily shows how different they are. Gottlieb et al. [2017] studies only the regression problem, whereas we considered a richer framework of Theorem 5.1 in their paper provided a risk bound that depends on the empirical risk We would be happy to update our literature review to make these connections.