Open Set Domain Adaptation by Extreme Value Theory

Recently, deep learning techniques have drawn a lot of attention in both academia and industry, due to their astounding performance in fields such as computer vision and natural language processing [1, 5, 10, 13, 16, 26, 31, 32, 35, 38]. However, one major disadvantage of deep learning is that neural networks generally require a large amount of training data to converge to the right solution and generalize. When the training data are insufficient, the model performance is usually adversely affected. Sometimes even if the training data are sufficient, the domain gap, i.e. the difference between data distributions, between the source domain (the data we train model on) and the target domain (the desired target task) may still contribute to low generalizability. This is because in conventional machine learning tasks, we usually assume that the training data distribution is the same as the testing data distribution. However, in real world, testing data are uncontrollable, and thus the difference between the source and the target domain can be substantial, which results in the overfitting problem, that is, the model does not generalize well to the testing set. In order to reduce the domain gap and better utilize the source domain knowledge, domain adaptation techniques have been proposed to resolve the issue. Domain adaptation assumes that the source domain has sufficient amount of labeled data to train a good model, while the desired target domain has insufficient amount of data to train the model.