Supplementary Material A More on Related Work

All of those works are for in-distribution fairness, and we investigate out-of-distribution fairness in this paper. In many real-world applications, distribution shifts are unavoidable. Thus the conditional distribution of the model's prediction also remains, as P Proposition B.2 (Transfer of fairness under subpopulation shift of sensitive attribute) . The proof is similar to the previous one. It suggests that encouraging fairness is able to alleviate spurious correlation.