Private Federated Learning with Autotuned Compression

Federated Learning (FL) is a form of distributed learning whereby a shared global model is trained collaboratively by many clients under the coordination of a central service provider. Often, clients are entities like mobile devices which may contain sensitive or personal user data. FL has a favorable construction for privacy-preserving machine learning, since user data never leaves the device. Building on top of this can provide strong trust models with rigorous user-level differential privacy (DP) guarantees [Dwork et al., 2010a], which has been studied extensively in the literature [Dwork et al., 2010b, McMahan et al., 2017b, 2022, Kairouz et al., 2021b]. More recently, it has become evident that secure aggregation (SecAgg) techniques [Bonawitz et al., 2016, Bell et al., 2020] are required to prevent honest-but-curious servers from breaching user privacy [Fowl et al., 2022, Hatamizadeh et al., 2022, Suliman and Leith, 2022]. Indeed, SecAgg and DP give a strong trust model for privacy-preserving FL [Kairouz et al., 2021a, Chen et al., 2022a, Agarwal et al., 2021, Chen et al., 2022b, Xu et al., 2023]. However, SecAgg can introduce significant communication burdens, especially in the large cohort setting which is preferred for DP. In the extreme, this can significantly limit the scalability of DP-FL with SecAgg. This motivated the study of privacy-utility-communication tradeoffs by Chen et al. [2022a], where they found that significant communication reductions could be attained essentially "for free" (see Figure 1) by using a variant of the Count Sketch linear data structure.