FedLab: A Flexible Federated Learning Framework

Federated learning (FL), proposed by Google at the very beginning [1], is recently a burgeoning research area of machine learning, which aims to protect individual data privacy in distributed machine learning process, especially in finance [2], smart healthcare [3, 4] and edge computing [5, 6]. Different from traditional data-centered distributed machine learning, participants in FL setting utilize localized data to train local model, then leverages specific strategies with other participants to acquire the final model collaboratively, avoiding direct data sharing behavior. Though it might differ in specific methodologies, current FL schemes can be summarized as repetition of training rounds, with each integrated by several basic steps: i) local update on client's model using their own localized data; ii) clients upload their local trained model parameters to server; iii) server performs aggregation strategy on collected clients' model parameters to obtain global model; iv) server selects a subset of clients and distributes the latest global model to them. Many FL researches try to improve algorithm effectiveness or efficiency on only one or more steps in this workflow with different scenarios: [7] suggests to add regularization term in step i) to achieve more robust convergence in heterogeneous settings; [8] applies gradient compression method in step ii) to reduce communication bandwidth; [9] tries to modify in step i), ii) and iii) for privacy-preserving purpose; [10] proposes better sample strategy in step iv) to address suboptimal result problem in Federated Multi-Task Learning. These indicate that the implementation of many FL algorithms only requires modification on several components of common workflow, without the necessity of repetitive implementation on basic FL workflow. The paradigm of FL and related research points are as depicted in figure 1.