An Information-theoretic Approach to Distribution Shifts

One of the most common assumptions for machine learning models is that the training and test data are independently and identically sampled (IID) from the same distribution. In practice, this assumption does not hold in many practical scenarios (Bengio et al., 2020).