Federated Multi-armed Bandits with Personalization

Federated learning (FL) is an emerging distributed machine learning paradigm that has many attractive properties. In particular, FL is motivated by the growing trend that massive amounts of real-world data are exogenously generated at edge devices, which are non-independent and identically distributed (non-IID) and highly imbalanced (Bonawitz et al., 2019). FL focuses on many clients collaboratively training a machine learning model under the coordination of a central server while keeping the local data private at each client (McMahan et al., 2017). Earlier FL approaches focus on training a single global model that can perform well on the aggregated global dataset. However, the performance of the FL-trained global model on an individual client dataset degrades dramatically when significant heterogeneity among the local datasets exists, which raises the concern of using one global model for all individual clients in edge inference. To address this issue, FL with personalization (Smith et al., 2017) has been proposed. Instead of learning a single global model, each device aims at learning a mixture of the global model and its own local model (Hanzely and Richtárik, 2020; Deng et al., 2020), which provides an explicit trade-off between the two potentially competing learning goals. While the main focus of the state of the art FL with personalization is on the supervised learning setting, we propose to extend its core principles to the multi-armed bandits (MAB) problem.