Multi-group Learning for Hierarchical Groups

In the classical statistical learning setup, the goal is to construct a predictor with high average accuracy. However, in many practical learning scenarios, an aggregate, on-average measure of performance is insufficient. In general, average-case performance can obscure performance for subgroups of examples -- a predictor that boasts 95% accuracy on average might only be 50% accurate on an important subgroup comprising 10% of the population. Such a subgroup might be difficult for a predictor trained for aggregate performance because it is not represented well during training (Oakden-Rayner et al., 2019) or it admits spurious correlations (Borkan et al., 2019). Recent work has shown that, in various learning domains, constructing a predictor that performs well on multiple subgroups is crucial. For instance, when fairness is a concern, a natural desideratum is that a predictor be accurate not only on average, but also conditional on possibly intersecting subgroups such as race and gender (Hardt et al., 2016; Diana et al., 2021). In medical imaging, a model might systematically err on rarer cancers to achieve a higher average accuracy, leading to possibly life-threatening predictions on individuals with the rare cancer (Oakden-Rayner et al., 2019).