
We consider two scalings of the first step learning rate ฮท . For small ฮท, we establish a Gaussian equivalence property for the trained feature map, and prove that the learned kernel improves upon the initial random feature model, but cannot defeat the best linear model on the input.