A Framework For Contrastive Self-Supervised Learning And Designing A New Approach

The first way we can characterize a contrastive self-supervised learning approach is by defining a data augmentation pipeline. A data augmentation pipeline A(x) applies a sequence of stochastic transformations to the same input. In deep learning, a data augmentation aims to build representations that are invariant to noise in the raw input. For example, the network should recognize the above pig as a pig even if it's rotated, or if the colors are gone or even if the pixels are "jittered" around. In contrastive learning, the data augmentation pipeline has a secondary goal which is to generate the anchor, positive and negative examples that will be fed to the encoder and will be used for extracting representations. CPC introduced a pipeline that applies transforms like color jitter, random greyscale, random flip, etc… but it also introduced a special transform that splits an image into overlaying sub patches.