On the Regularization Effect of Stochastic Gradient Descent applied to Least Squares

For symmetric matrices, this inequality has an extension to higher-order Sobolev spaces. This explains a (known) regularization phenomenon: an energy cascade from large singular values to small singular values smoothes.