Unsupervised Few-shot Learning via Deep Laplacian Eigenmaps

Few-shot learning (Fei-Fei et al., 2006) aims to learn a new classification or regression model on a novel task that is not seen during training, given only a few examples in the novel task. Existing few-shot learning methods either rely on episodic meta-learning (Finn et al., 2017, Snell et al., 2017) or standard pretraining (Chen et al., 2019, Tian et al., 2020b) in a supervised manner to extract transferrable knowledge to a new few-shot task. Unfortunately, these methods require many labeled meta-training samples. Acquiring a lot of labeled data is costly or even impossible in practice. Recently, several unsupervised meta-learning approaches have attempted to address this problem by constructing synthetic tasks on unlabeled meta-training data (Hsu et al., 2019, Khodadadeh et al., 2019, 2021) or meta-training on self-supervised pretrained features (Lee et al., 2021a). However, the performance of unsupervised meta-learning approaches is still far from their supervised counterparts. Empirical studies in supervised pretraining show that representation learning via grouping similar samples together (Chen et al., 2019, Dhillon et al., 2020, Laenen and Bertinetto, 2021, Tian et al., 2020b) outperforms a wide range of episodic meta-learning methods, where the definition of similar samples is given by class labels. The motivation of this study is to develop an unsupervised representation learning method by grouping unlabeled meta-training data without episodic training and close the performance gap between supervised and unsupervised few-shot learning. Contrastive self-supervised learning has shown remarkable success in learning representation from unlabeled data, which is competitive with supervised learning on multiple visual tasks (Hénaff et al., 2020, Tian et al., 2020a).