formalization of "how well a linear classifier explains the performance of a model " has many advantages over prior

We thank all the reviewers for their insightful comments and suggestions. Reviewer 1 We consider our mutual information framework to be a core contribution of our paper. "forget" the simple component even when trained to completion, provided it somehow learns the simple component Reviewer 2 Thank you for pointing out the relevant papers. We plan on including a separate section with such examples in the final version. Concretely, regarding "On the spectral bias of neural networks" [1]: They consider measuring "simplicity" via the Our metrics do not suffer from this issue - they are taken with respect to the true data distribution.