Algorithm 1 of PIC in a Algorithm 2 of supervised image like style

Algorithm 1 and 2 show the pseudocode of PIC and traditional supervised classification in a PyTorchlike style, respectively, which show that PIC can be easily adapted from supervised classification by only modifying a few lines of code. When we adopt the recent sampling strategy, those instance examples not included in the recent iterations will have zero gradient during training. Pre-training We follow the similar augmentation as Chen et al. [5] to adopt random resize and crop, random flip, strong color distortions, and Gaussian blur as the data augmentations, where the only difference is that we adopt the crop scale as 0.2 as Chen et al. [6]. We use Stochastic Gradient Descent (SGD) as our optimizer, with weight decay of 0.0001 and momentum as 0.9. We adopt a batch size of 512 in 8 GPUs with batch size per GPU as 64.