A.1 Scenario with only stragglers The hyperparameter settings for Sageflow are shown in Table 1. For the schemes ignore stragglers and wait for stragglers combined with FedAvg, we decayed the learning rate during training. For the FedAsync scheme of [7], we take a polynomial strategy with hyperparameters a= 0.5, α= 0.8, and decayed γ during training. A.2 Scenario with only adversaries Data poisoning and model poisoning attacks: Table 2 describes the hyperparameters for Sageflow with only adversaries, under data poisoning and model poisoning attacks. For RFA of [5], the maximum iteration is set to 10. In this setup, the learning rate is decayed for all three schemes (Sageflow, RFA, FedAvg).

We do so via novel data poisoning attacks, which weshowcorrespond torealistic privacyattacks.

For the latter, PT -based methods adaptively extract a matching width-based slimmed-down sub-model from the global model as a local model according to each client's budget, thus averting the requirements for public data. As with FedAvg, PT -based methods require the server to periodically communicate with the clients. Existing PT -based methods focus on how to extract width-based sub-models from the global model. DFKD methods are promising, which transfer knowledge from the teacher model to another student model without any real data. Existing DFKD methods can be broadly classified into non-adversarial and adversarial training methods. They take the quality and/or diversity of the synthetic data as important objectives.

We have 2 large datasets, HIGGS and Bosch (see reply to[R3]-1)). Table B highlights our differences.3) Motivation: We provide a strong attack as a tool for evaluating the9 robustnessoftreebasedmodels. MILP uses a thin wrapper around the Gurobi Solver.

For RFA of [5], the maximum iteration is set to 10. In this setup, the learning rate is decayed for all three schemes (Sageflow,RFA,FedAvg). The number of poisoned images inabatch is20, and we do not decay the learningratehere. Figure 1 shows theperformance under theno-scaled backdoor attack with only adversaries (nostragglers). The loss associated with a poisoned device increases if we increase the scale factor from 0.1 to 10.

In what follows we will call each experiment by its corresponding figure or table number for convenience. For the rotated/shifted MNIST images (Figure 8, 9), we use the Affine transformation function in the TorchVisionlibrary. In experiments (Table 2, 3, 4, 5), we use either or both of the Large (L) and Small (S) dataset for the standard benchmark vision data: MNIST, FMNIST, KMNIST, Omniglot, SVHN, CIFAR10, CIFAR100, CELEBA. For Figure 10, Table 3, the regularization coefficients for CAE, WAE are searched around 0.01 0.001, the noise level used in DAE is searched around0.1 0.01, and the regularization coefficient andλforSPAEandNRAE aresearched around0.001 Ontheother hand, the runtimes of our algorithms are comparable with other existing methods.

Locally interpretable model agnostic explanations (LIME) method is one of the most popular methods used to explain black-box models at a per example level.