Effective Sharpness A ware Minimization Requires Layerwise Perturbation Scaling

Our findings reveal that the dynamics of standard SAM effectively reduce to applying SAM solely in the last layer in wide neural networks, even with optimal hyperparameters.