Stochastic Unrolled Federated Learning

Federated learning is a distributed learning paradigm in which a set of agents aim to collaboratively train a global statistical model. Due to privacy considerations, rather than sharing local training data, agents are incentivized to communicate either their local models or gradient information to their network. Yet, communication efficiency is a crucial factor to consider, as the network could potentially incur latency, congestion, and failures. A growing body of work, e.g., [Lian et al., 2015, McMahan et al., 2016, Li et al., 2020] has deployed a server in the network to facilitate reaching consensus among the agents, which despite its efficiency, creates a communication bottleneck at the server. On the other hand, another line of work that traces back to decentralized optimization [Nedic and Ozdaglar, 2009, Wei and Ozdaglar, 2012, Wu et al., 2017] has been investigated to design federated learning frameworks without a central server, compromising communication efficiency and convergence rates [Vanhaesebrouck et al., 2017, Liu et al., 2022a,b]. Indeed, the two categories have been enriched by the advances in iterative optimization algorithms and, in particular, stochastic gradient descent (SGD) and its variants. While learning frameworks have significantly benefited from well-crafted optimization algorithms, the converse has also been made possible due to algorithm unrolling [Chen et al., 2021b, Monga et al., 2021].