Unsupervised Pre-training

Pre-training general-purpose visual features with convolutional neural networks without relying on annotations is a challenging and important task. Most recent efforts in unsupervised feature learning have focused on either small or highly curated datasets like ImageNet, whereas using non-curated raw datasets was found to decrease the feature quality when evaluated on a transfer task. Suppose you want to tackle a complex task for which you don't have much labeled training data, but unfortunately you cannot find a model trained (pretrained model) on a similar task. It is often rather cheap to gather unlabeled training examples, but quite expensive to label them. An autoencoder is a type of artificial neural network used to learn data encodings in an unsupervised manner.