Appendix

Section A provides more details about our algorithms. In particular, Section A.1 describes the Section A.2 describes the REx implementation of invariance in Section A.3 discusses the two implementation of the self-training used in our experiments: Section B gives the preliminaries of group theory. Section C gives the full proof of our theorem. Section E shows more comparisons and standard deviations on WILDS 2.0 benchmark [ The setup instructions and commands used in our experiments are included in the README.md Self-training loss α Self-training loss weight β Invariance loss weight Symbol in Theory G Group of semantics X Feature space H Subgroup of G that transforms environmental feature e g x Group action Table A1: List of abbreviations and symbols used in the paper. 1 A Additional Details on Algorithm REx circumvents the challenging bi-level optimization from line 2 of Eq. (2) by adding the following Please refer to [8] for a theoretical explanation.