Review for NeurIPS paper: Batch normalization provably avoids ranks collapse for randomly initialised deep networks

Weaknesses: While there are not many technical or experimental weaknesses in this paper, I wonder whether rank preserving transformations are important in other learning models - say linear ones or kernel machines, etc. It could be the case that this is a phenomenon exclusive to deep networks and other models are not relevant. Another issue is that in the case of binary classification one could still perform the task when rank collapse happens, as long as the relevant discriminatory signal is captured by the principal direction that the data is collapsed to. I would like to know if the authors agree or disagree with this hypothetical. Finally I do not think the authors address the case where the networks are overparameterized and d N in each layer.