JointContrastiveLearningwithInfinitePossibilities--SupplementaryMaterials

For all the experiments, we generate augmentations in the same way as in MoCo v2 [1] for pretraining. The learning rate is set tolr = 0.1 and is gradually annealed following a cosine decay schedule [3]. For linear classification, all models are trained for 100 epochs with alearning rate oflr = 10.0. For each image, we randomly generate 32 augmented images and feed these images into the pre-trained network toextract features. The feature vectors are`2 normalized before computing similarities and variances.