Does Distributionally Robust Supervised Learning Give Robust Classifiers?

Distributionally Robust Supervised Learning (DRSL) is necessary for building reliable machine learning systems. When machine learning is deployed in the real world, its performance can be significantly degraded because test data may follow a different distribution from training data. Previous DRSL explicitly considers the worst-case distribution shift by minimizing the adver-sarially reweighted training loss. In this paper, we theoretically analyze the previous DRSL in a classification scenario. We reveal a rather surprising fact that the previous DRSL ends up giving classifiers optimal for the training distribution even though it is designed to be robust to change from the training distribution. Motivated by our analysis, we also propose novel DRSL that overcomes this limitation. We establish its convergence property and demonstrate its effectiveness through experiments.