We propose a few-shot learning method for detecting out-of-distribution (OOD) samples from classes that are unseen during training while classifying samples from seen classes using only a few labeled examples. For detecting unseen classes while generalizing to new samples of known classes, we synthesize fake samples, i.e., OOD samples, but that resemble in-distribution samples, and use them along with real samples. Our approach is based on an extension of model-agnostic meta learning (MAML) and is denoted as OOD-MAML, which not only learns a model initialization but also the initial fake samples across tasks. The learned initial fake samples can be used to quickly adapt to new tasks to form task-specific fake samples with only one or a few gradient update steps using MAML. For testing, OOD-MAML converts a K-shot N-way classification task into N sub-tasks of K-shot OOD detection with respect to each class. The joint analysis of N sub-tasks facilitates simultaneous classification and OOD detection and, furthermore, offers an advantage, in that it does not require re-training when the number of classes for a test task differs from that for training tasks; it is sufficient to simply assume as many sub-tasks as the number of classes for the test task. We also demonstrate the effective performance of OOD-MAML over benchmark datasets.

Omniglot (Lake et al., 2015) is a dataset of handwritten characters and contains 20 examples of 1623 characters. Omniglot is the most commonly used dataset in few-shot learning, and its images are resized to 28 28 (Finn et al., 2017; Santoro et al., 2016; Snell et al., 2017; Sung et al., 2018; Koch et al., 2015). As in other studies, we randomly select 1200 characters for meta-training and use the remaining for meta-testing. It contains a total of 60K images of 100 different classes, each of which comprises 600 RGB images. Ravi and Larochelle (2016) presented the protocol for mini ImageNet as per which all the images are downsampled to 84 84 and are divided into 64 classes for meta-training, 16 classes for meta-validation, and 20 for meta-testing. We followed this protocol but did not use the meta-validation set.

For example, let us suppose that a child learns the alphabets'A ' and'B'

For example, let us suppose that a child learns the alphabets'A ' and'B'

We deeply appreciate the reviewers' careful comments. We hope all concerns can be resolved through our clarifications. Q: I'd recommend picking an OOD detection threshould at 95% TPR for a more even comparison. A: Thank you for your great suggestion. Previously we set the threshold at 0.5 as a default value for binary classification.

Correctness: In Table 1, baseline methods are thresholded at a 95% TPR, while the proposed method and its variants are claimed to be threshold-agnostic: From section 3.3 it appears that the threshold is manually set to be 0.5, so they are not really threshold agnostic. It feels likely to me that there might be situations where picking thresholds with different criteria for comparative methods might lead to an unfair assessment. I'd recommend picking an OOD-detection threshold (on the maximum softmax values for class 1 across all tasks, for example) also at 95% TPR for a more even comparison. The experiment in Section 4.4 feels a bit anecdotal due to the particular example studied. Appendix D studies the effect of the adversarial adaptation, and while the text says random-(ini)-OOD outperforms random-OOD, the table seems to show the opposite trend (a typo perhaps?), which would indicate the adversarial adaptation did not help.

This paper presents a method for performing meta-learning and OOD detection. The reviewers agreed that this paper meets the bar for acceptance. For the camera ready, the authors are encouraged to address the reviewer's feedback that was discussed in the author response, address the other points of feedback (e.g.

We propose a few-shot learning method for detecting out-of-distribution (OOD) samples from classes that are unseen during training while classifying samples from seen classes using only a few labeled examples. For detecting unseen classes while generalizing to new samples of known classes, we synthesize fake samples, i.e., OOD samples, but that resemble in-distribution samples, and use them along with real samples. Our approach is based on an extension of model-agnostic meta learning (MAML) and is denoted as OOD-MAML, which not only learns a model initialization but also the initial fake samples across tasks. The learned initial fake samples can be used to quickly adapt to new tasks to form task-specific fake samples with only one or a few gradient update steps using MAML. For testing, OOD-MAML converts a K-shot N-way classification task into N sub-tasks of K-shot OOD detection with respect to each class.