When Does Feature Learning Happen? Perspective from an Analytically Solvable Model