Diving into the shallows: a computational perspective on large-scale shallow learning

Our analysis demonstrates that only a vanishingly small fraction of the function space is reachable after a polynomial number of gradient descent iterations.