Adaptive Prototypical Networks

Supervised Learning with deep networks, although performs very well in domains with large amounts of data, it performs rather poorly in domains where the data distribution is scarce or has a long tail. With the aim of few-shot learning, this issue is mitigated quite convincingly by meta-learning algorithms. One such meta-learning algorithm is Prototypical networks Snell et al. [2017]. The working of Prototypical networks, like other metric-based meta-learning algorithms, is based on learning an embedding function or an embedding space such that a non-parametric learner can easily classify the images in the learned embedding space. In particular, prototypical networks learn an embedding function with euclidean distance as a similarity metric between the query embedding and the class prototype, which is the mean vector of the support set embeddings. This results in similar-looking images getting embedded close to each other in the embedding space and forming clusters. However, along with clustering of similar-looking images, it is more or less equally desirable to have separation of embeddings based on the distinct class labels when it comes to the task of classification. In prototypical networks, each image in a task is embedded independently, and hence there is no information shared between the embeddings regarding the similarity of classes. Thus, in the presence of similar classes or classes with similar features (for, e.g. the English alphabet'p' and Greek alphabet 'ρ') in a particular task, the embeddings from the similar classes lie very close