Enhancing Sharpness-Aware Optimization Through Variance Suppression

Sharpness-aware minimization (SAM) has well documented merits in enhancing generalization of deep neural networks, even without sizable data augmentation. Embracing the geometry of the loss function, where neighborhoods of'flat minima' heighten generalization ability, SAM seeks'flat valleys' by minimizing the maximum loss caused by an perturbing parameters within the neighborhood.Although critical to account for sharpness of the loss function, such an ' adversary' can curtail the outmost level of generalization. The novel approach of this contribution fosters stabilization of adversaries through (VaSSO) to avoid such friendliness.