Smoothing the Landscape Boosts the Signal for SGD Optimal Sample Complexity for Learning Single Index Models

We focus on the task of learning a single index model σ(w x) with respect to the isotropic Gaussian distribution in d dimensions. Prior work has shown that the sample complexity of learning w is governed by the information exponent k of the link function σ, which is defined as the index of the first nonzero Hermite coefficient of σ.