Federated Learning on Stochastic Neural Networks

Original Manuscript Submitted: 05/05/2025; Final Draft Received: mm/dd/yyyy Federated learning is a machine learning paradigm that leverages edge computing on client devices to optimize models while maintaining user privacy by ensuring that local data remains on the device. However, since all data is collected by clients, federated learning is susceptible to latent noise in local datasets. Factors such as limited measurement capabilities or human errors may introduce inaccuracies in client data. T o address this challenge, we propose the use of a stochastic neural network as the local model within the federated learning framework. Stochastic neural networks not only facilitate the estimation of the true underlying states of the data but also enable the quantification of latent noise. We refer to our federated learning approach, which incorporates stochastic neural networks as local models, as Federated stochastic neural networks. We will present numerical experiments demonstrating the performance and effectiveness of our method, particularly in handling non-independent and identically distributed data. KEY WORDS: Machine Learning, Federated Learning, Neural Network 1. INTRODUCTION The fundamental principles of federated learning can be traced back to earlier advancements in distributed computing and privacy-preserving machine learning techniques. Before federated learning was introduced in McMahan et al. (2016), distributed machine learning primarily focused on executing training processes in parallel across multiple nodes within a data center. Notable frameworks, such as MapReduce (Dean and Ghemawat (2004)) and AllReduce, were designed to aggregate data from different computational units, perform global aggregation using predefined operators, and subsequently redistribute the outcomes to all participating units.