Federated Learning (FL) is currently one of the most popular technologies in the field of Artificial Intelligence (AI) due to its collaborative learning and ability to preserve client privacy. However, it faces challenges such as non-identically and non-independently distributed (non-IID) and data with imbalanced labels among local clients. To address these limitations, the research community has explored various approaches such as using local model parameters, federated generative adversarial learning, and federated representation learning. In our study, we propose a novel Clustered FedStack framework based on the previously published Stacked Federated Learning (FedStack) framework. The local clients send their model predictions and output layer weights to a server, which then builds a robust global model. This global model clusters the local clients based on their output layer weights using a clustering mechanism. We adopt three clustering mechanisms, namely K-Means, Agglomerative, and Gaussian Mixture Models, into the framework and evaluate their performance. We use Bayesian Information Criterion (BIC) with the maximum likelihood function to determine the number of clusters. The Clustered FedStack models outperform baseline models with clustering mechanisms. To estimate the convergence of our proposed framework, we use Cyclical learning rates.

Over the past couple of days, I've been thinking about the late, great 30 Rock--in particular, an episode from Season 1. Dennis Duffy (Dean Winters), the marvelously terrible boyfriend of Liz Lemon (Tina Fey), is a bit of a technology entrepreneur--by which I mean he is the Beeper King, the last beeper salesman on Manhattan. "Which is cool," his then-girlfriend Liz Lemon (Tina Fey) tells a skeptical friend. But when he tries to sell beepers to her staff of TV writers, she loses it and tells him to leave. "You work in a business. Businesspeople need beepers," he insists.