Federated Split Learning with Improved Communication and Storage Efficiency

--Federated learning (FL) is one of the popular distributed machine learning (ML) solutions but incurs significant communication and computation costs at edge devices. Federated split learning (FSL) can train sub-models in parallel and reduce the computational burden of edge devices by splitting the model architecture. However, it still requires a high communication overhead due to transmitting the smashed data and gradients between clients and the server in every global round. Furthermore, the server must maintain separate partial models for every client, leading to a significant storage requirement. T o address these challenges, this paper proposes a novel communication and storage efficient federated split learning method, termed CSE-FSL, which utilizes an auxiliary network to locally update the weights of the clients while keeping a single model at the server, hence avoiding frequent transmissions of gradients from the server and greatly reducing the storage requirement of the server . Additionally, a new model update method of transmitting the smashed data in selected epochs can reduce the amount of smashed data sent from the clients. We provide a theoretical analysis of CSE-FSL, rigorously guaranteeing its convergence under non-convex loss functions. The extensive experimental results further indicate that CSE-FSL achieves a significant communication reduction over existing FSL solutions using real-world FL tasks. S an emerging distributed machine learning (ML) paradigm, federated learning (FL) [2] enables distributed clients to collaboratively train ML models without uploading their sensitive data to a central server. While FL addresses privacy concerns by keeping data localized, most existing FL approaches assume that clients possess sufficient computational and storage resources to perform local updates on large (potentially deep) models. However, this assumption breaks down in scenarios where clients, such as mobile and Internet-of-Things (IoT) devices, are resource-constrained. Consequently, these clients struggle to handle the heavy computational and storage demands of training deep ML models, making FL impractical in such settings. To address this issue, split learning (SL) [3]-[5] is proposed.