The Crucial Role of Normalization in Sharpness-Aware Minimization Yan Dai

Sharpness-A ware Minimization (SAM) is a recently proposed gradient-based optimizer (Foret et al., ICLR 2021) that greatly improves the prediction performance of deep neural networks.