Personalized Federated Learning with Attention-based Client Selection

Federated learning is a collaborative learning paradigm that allows multiple clients to work together while ensuring the preservation of their privacy Yang et al. (2019a). By leveraging the collective knowledge and data from all participating clients, federated learning aims to achieve better learning performance compared with individual client efforts McMahan et al. (2017). This collaborative nature has made federated learning increasingly popular, finding numerous practical applications where data decentralization and privacy are paramount. The privacy-preserving solutions offered by federated learning have found extensive applications in domains such as healthcare, smart cities, and finance Zheng et al. (2022); Yang et al. (2019b); Xu et al. (2021). However, the effectiveness of the collaborative approach in federated learning is highly dependent on the distribution of data among the clients. While federated learning performs exceptionally well when data distribution among clients is independent and identically distributed (IID), this is not the case in many real-world scenarios Kairouz et al. (2021). When the global model, which is collectively trained across decentralized clients, encounters diverse datasets with varying statistical characteristics, it may face challenges in effectively generalizing to the unique local data of each client Zhao et al. (2018); Jiang et al. (2019). The performance of the global model may be suboptimal on certain clients' data due to differences in data distributions and patterns. This limitation becomes more pronounced as the diversity among local data from different clients continues to increase Deng et al. (2020).