b432f34c5a997c8e7c806a895ecc5e25-AuthorFeedback.pdf

We thank all the reviewers for their insightful comments and suggestions. We consider our mutual information framework to be a core contribution of our paper. "forget" the simple component even when trained to completion, provided it somehow learns the simple component Thank you for pointing out the relevant papers. We plan on including a separate section with such examples in the final version. Concretely, regarding "On the spectral bias of neural networks" [1]: They consider measuring "simplicity" via the Our metrics do not suffer from this issue - they are taken with respect to the true data distribution.