Blessing of Depth in Linear Regression: Deeper Models Have Flatter Landscape Around the True Solution

For instance, the existence of spurious local minima may not pose any issue if an optimization algorithm can avoid them efficiently.