Note thatppos is task-specific; here we use the class oracle,i.e. the ImageNet-100 labels,todefinethepositivesamples. In Figure 1, we plot theproxy task performance, i.e. the percentage of queries where the key is ranked over all negatives, across training for MoCo [19], MoCo-v2 [10] and some variants inbetween. As mentioned above, all results in Figure1areforthesameτ =0.2. Ablations showed that this yields at best performance as good as mixingwiththequery,butonaverageabout0.1-0.2%lower. This weighing scheme also resulted in slightly inferior results.

Wetherefore startbydelving deeper intoatop-performing frameworkandshow evidence that harder negatives are needed to facilitate better and faster learning.

Our rule may be understood as ageneralization of contrastive Hebbian learning to meta-learning and notably, it neither requires computing second derivativesnorgoing backwardsintime,twocharacteristic features of previous gradient-based methods that are hard to conceive in physicalneuralcircuits.

The author contributed to this paper during an internship at the Max Planck Institute for Software Systems.