Third, weintroduce acomplexitymeasure (seeDefinition 5)thatcharacterizes theoptimal sample complexity of learning in settings (ii) and (iii) above, and we give optimal algorithms for these settings. Finally,wealso provide adaptivelearning algorithms that interpolate between settings (i) and (ii), i.e., whenh is partiallyinvariant.

Although significant efforts have been made into tackling statistical data heterogeneity,the diversity in the processing capabilities andnetworkbandwidth ofclients,termedassystemheterogeneity,hasremained largelyunexplored.

We here argue that popular benchmarks to measure model robustness againstcommon corruptions (likeImageNet-C) underestimate model robustness in many (but not all) application scenarios. The key insight is that inmanyscenarios, multiple unlabeled examples ofthe corruptions are available and can be used for unsupervised online adaptation.