Comprehensive Privacy Analysis of Deep Learning: Stand-alone and Federated Learning under Passive and Active White-box Inference Attacks

Abstract--Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We perform a comprehensive analysis of white-box privacy inference attacks on deep learning models. We measure the privacy leakage by leveraging the final model parameters as well as the parameter updates during the training and finetuning processes.We design the attacks in the stand-alone and federated settings, with respect to passive and active inference attackers, and assuming different adversary prior knowledge. We evaluate our novel white-box membership inference attacks against deep learning algorithms to measure their training data membership leakage. We show that a straightforward extension of the known black-box attacks to the white-box setting (through analyzing the outputs of activation functions) is ineffective. We therefore design new algorithms tailored to the white-box setting by exploiting the privacy vulnerabilities of the stochastic gradient descent algorithm, widely used to train deep neural networks. We show that even well-generalized models are significantly susceptible towhite-box membership inference attacks, by analyzing state-of-the-art pre-trained and publicly available models for the CIFAR dataset. We also show how adversarial participants of a federated learning setting can run active membership inference attacks against other participants, even when the global model achieves high prediction accuracies. I. INTRODUCTION Machine learning models based on deep neural networks have been shown to have significantly high generalization accuracies for various learning tasks, from image and speech recognition to generating realistic-looking data. This success has led to many applications and services that use deep learning algorithms on large-dimension (and potentially sensitive) userdata, including user speeches, images, and medical, financial, social, and location data points. The crucial question we ask in this paper is the following: How much is the privacy risks of deep learning for individuals whose data is used as part of the training set? In other words, how much is the information leakage of deep learning algorithms about their individual training data samples? We define privacy-sensitive leakage of a model about a set of target trained data record(s) as the information that an adversary can infer about the data records, that could not have been inferred from similar models not using the target data records. This distinguishes between the information that we can learn from the model about the population and the information that it leaks about particular samples from the population that were in its training set. The former indicates utility gain, and the later reflects privacy loss. We design inference attacks to quantify such privacy leakage. Theinference attacks fall into two fundamental and related categories: tracing (a.k.a.