A Generalized Unbiased Risk Estimator for Learning with Augmented Classes

Machine learning approaches have achieved great performance on a variety of tasks, and most of them focus on the stationary learning environment. However, the learning environment in many real-world scenarios could be open and change gradually, which requires the learning approaches to have the ability of handling the distribution change in the non-stationary environment [1-4]. This paper considers a specific problem where the class distribution changes from the training phase to the test phase, called learning with augmented classes (LAC). In LAC, some augmented classes unobserved in the training phase might emerge in the test phase. In order to make accurate and reliable predictions, the learning model is required to distinguish augmented classes and keep good generalization performance over the test distribution. The major difficulty in LAC is how to exploit the relationships between known and augmented classes. To overcome this difficulty, various learning methods have been proposed. For example, by learning a compact geometric description of known classes to distinguish augmented classes that are far away from the description, the anomaly detection or novelty detection methods can be used (e.g., iForest [5], one-class SVM [6, 7], and kernel density estimation [8, 9]). By exploiting unlabeled data with the low-density separation assumption to adjust the classification decision boundary [10], the performance of LAC can be empirically improved.