Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks

Knowledge distillation Soft labeling becomes a common output regularization (Hinton et al., 2015), a kind of soft labeling (Yuan et al., for generalization and model compression 2019), can compress the size of a teacher model, or improve of deep neural networks. However, the effect the accuracy of its student networks (Xie et al., 2019). of soft labeling on out-of-distribution (OOD) detection, which is an important topic of machine Despite the popularity of soft labeling, how soft labeling learning safety, is not explored. In this study, we affects OOD detection of DNNs has not been explored. In show that soft labeling can determine OOD detection this study, we assume that regularizing predictions on incorrect performance. Specifically, how to regularize classes by soft labeling determines OOD detection outputs of incorrect classes by soft labeling can performance of DNNs. We analyze and empirically verify deteriorate or improve OOD detection. Based on our assumption, based on two major results: a) label the empirical results, we postulate a future work smoothing deteriorates OOD detection of DNNs, and b) for OOD-robust DNNs: a proper output regularization soft labels, generated by a teacher model, distill OOD detection by soft labeling can construct OOD-robust performance into its student models. In particular, DNNs without additional training of OOD samples the degraded test accuracy of a teacher model with outlier or modifying the models, while improving exposure is recovered or improved in its student models, classification accuracy.