Eavesdrop the Composition Proportion of Training Labels in Federated Learning

--Federated learning (FL) has recently emerged as a new form of collaborative machine learning, where a common model can be learned while keeping all the training data on local devices. Although it is designed for enhancing the data privacy, we demonstrated in this paper a new direction in inference attacks in the context of FL, where valuable information about training data can be obtained by adversaries with very limited power . In particular, we proposed three new types of attacks to exploit this vulnerability. The first type of attack, Class Sniffing, can detect whether a certain label appears in training. The other two types of attacks can determine the quantity of each label, i.e., Quantity Inference attack determines the composition proportion of the training label owned by the selected clients in a single round, while Whole Determination attack determines that of the whole training process. We evaluated our attacks on a variety of tasks and datasets with different settings, and the corresponding results showed that our attacks work well generally. Finally, we analyzed the impact of major hyper-parameters to our attacks and discussed possible defenses. The emergence of federated learning (FL) enables multiple devices to learn a common model while keeping all the training data on their own devices. It allows for less resource consumption on the cloud and ensures the privacy at the same time. Multiple applications have benefited from FL, including mobile phones [1, 2, 3], wearable devices [4, 5], autonomous vehicles [6, 7], etc. In standard federated learning, all participants are required to train their local models. A random subset of clients will be selected each round, who will upload their gradient updates to the central server. Similar FL architectures can be found in [8, 9, 10, 11, 12, 13]. One interesting question here is about the security and privacy implication in the FL training process. Any characteristic of clients' private data needs to be protected carefully since it may reveal some important private information about the training data - e.g., the distribution of labels might show the diversity of participants. Similarly, what the training data consists of is also what attackers want to explore, i.e., can they determine the quantity proportion of different labels in the whole training dataset during the training process?