A Unified Mixture-View Framework for Unsupervised Representation Learning

Unsupervised representational learning is now on the very rim to take over supervised representation learning. It is supposed to be a perfect solver for real-world scenarios full of unlabeled data. Among them, self-supervised learning has drawn the most attention for its good data efficiency and generalizability. Self-supervised learning typically involves a proxy task to learn discriminative representations from self-derived labels. Among all manners of these proxy tasks [13, 18, 29, 33, 36], instance discrimination [31, 49], known as contrastive representation learning, has emerged as the most effective paradigm. Its subsequent methods [7, 22, 24, 41, 59] have greatly reduced the gap between unsupervised and supervised learning. Specifically, instance discrimination features a Single Instance Multi-view (SIM) paradigm to separate different instances. It seeks to narrow the distance among multiple views of the same instance (e.g. an image), which are typically yielded from vanilla data augmentation policies like color jittering, cropping, resizing, applying Gaussian noise.