f21e255f89e0f258accbe4e984eef486-Supplemental.pdf

Mathematically characterizing the implicit regularization induced by gradientbased optimization is a longstanding pursuit in the theory of deep learning. A widespread hope isthatacharacterization based onminimization ofnorms may apply, and a standard test-bed for studying this prospect is matrix factorization (matrix completion via linear neural networks). It is an open question whether normscanexplaintheimplicit regularization inmatrixfactorization.