Statistical Estimation and Inference via Local SGD in Federated Learning

Federated Learning is a novel distributed computing paradigm for collaboratively training a global model from data that remote clients hold [McMahan et al., 2017]. The clients can only cooperate with a central server (e.g., service provider) to train the global model without sharing local datasets. Thus, federated learning can protect sensitive information that data often contain, such as personal identity information and state of health information, from unauthorized access of service providers. The challenge arises when limited data access together with memory constraints, communication budget, and computation restrictions make the traditional statistical estimation and inference methods [Li et al., 2020b, Fan et al., 2021] no longer applicable in the federated learning scenario. A typical federated learning system considers a pool of K clients, in which the k-th client has a local dataset consisting of i.i.d.