Federated Learning with Server Learning: Enhancing Performance for Non-IID Data

Federated Learning (FL) has emerged as a means of distributed learning using local data stored at clients with a coordinating server. Recent studies showed that FL can suffer from poor performance and slower convergence when training data at clients are not independent and identically distributed. Here we consider a new complementary approach to mitigating this performance degradation by allowing the server to perform auxiliary learning from a small dataset. Our analysis and experiments show that this new approach can achieve significant improvements in both model accuracy and convergence time even when the server dataset is small and its distribution differs from that of the aggregated data from all clients. Federated Learning (FL) is a recent paradigm in which multiple clients collaborate under the coordination of a central server to train machine learning (ML) models [13]. A key advantage is that clients need not send their local data to any central sever or share their data with each other. Performing learning where the data is generated (or collected) is becoming necessary as a large and growing amount of data is created at the network edge and cannot all be forwarded to any central location due to many factors such as network capacity constraints, latency requirements, and data privacy concerns [4]. In its basic form, FL trains a global model for all clients based on the following high-level iterative procedure. At each global round: 1) the central server selects a subset of clients and shares the current global model with them, 2) each selected client updates the model using only its local data and forwards the updated model to the central server, and 3) the central server aggregates the updated local models from the clients to update the global model.