Learning in the Presence of Low-dimensional Structure: A Spiked Random Matrix Perspective

Our results demonstrate that both kernel methods and neural networks benefit from low-dimensional structures in the data. Further, since k p by definition, neural networks can adapt to such structures more effectively.