TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels

Federated learning is a newly emerging paradigm for machine learning where multiple data holders (clients) collaborate to train a model on their combined dataset. Clients only share partially trained models and other statistics computed from their dataset, keeping their raw data local and private [McMahan et al., 2017, Kairouz et al., 2021]. By obviating the need for a third party to collect and store clients' data, federated learning has several advantages over the classical, centralized paradigm [Dean et al., 2012, Iandola et al., 2016, Goyal et al., 2017]: it ensures clients' consent is tied to the specific task at hand by requiring active participation of the clients in training, confers some basic level of privacy, and has the potential to make machine learning more participatory in general [Kulynych et al., 2020, Jones and Tonetti, 2020]. Further, widespread legislation of data portability and privacy requirements (such as GDPR and CCPA) might even make federated learning a necessity [Pentland et al., 2021]. Collaboration among clients is most attractive when clients have very different subsets of the combined dataset (data heterogeneity).