Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning

In many settings, it is infeasible to transfer an entire dataset to a centralized cloud for downstream analysis, either due to practical constraints such as high communication cost or latency, or to maintain user privacy and security [11]. This has led to the deployment of distributed machine learning and deep-learning techniques where computation is performed collaboratively by a set of clients, each close to its own data source. Federated learning has emerged as a popular technique in this space, which performs iterative collaborative training of a global machine learning model over data distributed over a large number of clients without sending raw data over the network. The more commonly studied approach of federated learning is horizontal federated learning. In horizontal federated learning, the clients' datasets share the same set of features, but each client holds only a subset of the sample space, i.e., the data is horizontally partitioned among clients [11, 15, 21]. In this setting, the clients train a copy of a model on their local datasets for a few iterations and then communicate their updates in the form of model weights or gradients directly to a centralized parameter server. The parameter server then creates the centralized model by aggregating the individual client updates, and the process is repeated until the desired convergence criterion is met. Another scenario that arises in federated learning is when clients have different sets of features, but there is a sizable overlap in the sample ID space among their datasets [33].