Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape

Due to the poor bandwidth throttling especially global server and cooperatively train one model on the global server, it adopts multiple local training with privacy protection. Due to the multiple local and partial participation to mitigate the communication bottleneck updates and the isolated non-iid dataset, clients to a greater extent. With extensive studies of FL, are prone to overfit into their own optima, which theoretical analysis reveals that the major influence on limiting extremely deviates from the global objective and the performance of FL is client drifts, whose essence significantly undermines the performance. Most is that the inconsistent local optima deviate from the global previous works only focus on enhancing the consistency objective on the heterogeneous dataset (Karimireddy et al., between the local and global objectives 2020; Woodworth et al., 2020; Li et al., 2020b; Kairouz to alleviate this prejudicial client drifts from the et al., 2021). Yang et al. (2021) theoretically demonstrate perspective of the optimization view, whose performance that the performance of the classical FedAvg method suffers would be prominently deteriorated on from the length of local updates and the number of the high heterogeneity. In this work, we propose a partial participation multiplied by the constant upper bound novel and general algorithm FedSMOO by jointly of the variance of the heterogeneous gradient, which contributes considering the optimization and generalization as the dominant term of the convergence rate. This targets to efficiently improve the performance in divergence would be extremely multiplied by both increasing FL. Concretely, FedSMOO adopts a dynamic regularizer the local interval and reducing the participation ratio.