Review for NeurIPS paper: Robust Recovery via Implicit Bias of Discrepant Learning Rates for Double Over-parameterization
–Neural Information Processing Systems
Additional Feedback: To be honest, I find the term "double overparametrization" a bit strange. I would still call it simply "overparametrization". Perhaps, the authors could think about this point and potentially adjust. I would suggest that the authors briefly discuss the following point which is sometimes overlooked when discussing implicit bias of gradient descent in the context of low rank matrix recovery. When additional restricting to positive semidefinite matrices it turns out that the original low rank matrix is often the UNIQUE solution to the linear equation y A(X) that is positive semidefinite, see the paper "Implicit regularization and solution uniqueness in over-parameterized matrix sensing" by Geyer et al., arxiv:806.02046,
Neural Information Processing Systems
Feb-6-2025, 09:27:12 GMT
- Technology: