MNIST Partition 1 MNIST Partition 2 MNIST Partition 2 (Permuted Labels) CIFAR - 10 Equally Initialized Parameters Client Gradient Update Label Heterogeneous Domain Heterogeneous! Local Training Steps