Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks

Adding depth not only improves performance on these tasks but also acts as an accelerative pre-conditioning that further enhances this bias towards low-rankedness.