Passing gradient is a widely used scheme in modern multi-node learning system (e.g, distributed training, collaborative learning). In a long time, people used to believe that gradients are safe to share: i.e, the training set will not be leaked by gradient sharing. However, in this paper, we show that we can obtain the private training set from the publicly shared gradients. The leaking only takes few gradient steps to process and can obtain the original training set instead of look-alike alternatives. We name this leakage as \textit{deep leakage from gradient} and practically validate the effectiveness of our algorithm on both computer vision and natural language processing tasks. We empirically show that our attack is much stronger than previous approaches and thereby and raise people's awareness to rethink the gradients' safety. We also discuss some possible strategies to defend this deep leakage.

Neural Information Processing Systems http://nips.cc/

Distributed training becomes necessary to speedup training on large-scale datasets.

Elegant approach - The approach, unlike [27] is much simpler and requires weaker assumptions to reconstruct the input data. Major concerns: a. Attack model / gradient computation - The authors look at the specific case of reconstructing raw private inputs resulting from gradients resulting from a single iteration computed on a small batch of images. The attack model assumes these are shared to the adversary. In this particular case, I'm skeptical of the effectiveness of the proposed attack. Missing details / writing I strongly recommend the authors to make more passes to fix typos/gramma and add many missing details that makes the findings unclear: - Implementation: * L135: CIFAR CIFAR10 or CIFAR100?

The paper presents an attack against federated learning algorithms and shows that when certain conditions apply, it may be possible to reconstruct the raw data from the gradients. This is an interesting observation. Federated learning, despite not having any formal privacy guarantees, is gaining popularity in corporates that operate on large amounts of data. In some cases, it might be used under the assumption that it provides privacy. Therefore, showing that this feeling is wrong may have real world impact.

Passing gradient is a widely used scheme in modern multi-node learning system (e.g, distributed training, collaborative learning). In a long time, people used to believe that gradients are safe to share: i.e, the training set will not be leaked by gradient sharing. However, in this paper, we show that we can obtain the private training set from the publicly shared gradients. The leaking only takes few gradient steps to process and can obtain the original training set instead of look-alike alternatives. We name this leakage as \textit{deep leakage from gradient} and practically validate the effectiveness of our algorithm on both computer vision and natural language processing tasks. We empirically show that our attack is much stronger than previous approaches and thereby and raise people's awareness to rethink the gradients' safety.

Federated Learning (FL) has emerged as a machine learning approach able to preserve the privacy of user's data. Applying FL, clients train machine learning models on a local dataset and a central server aggregates the learned parameters coming from the clients, training a global machine learning model without sharing user's data. However, the state-of-the-art shows several approaches to promote attacks on FL systems. For instance, inverting or leaking gradient attacks can find, with high precision, the local dataset used during the training phase of the FL. This paper presents an approach, called Deep Leakage from Gradients with Feedback Blending (DLG-FB), which is able to improve the inverting gradient attack, considering the spatial correlation that typically exists in batches of images. The performed evaluation shows an improvement of 19.18% and 48,82% in terms of attack success rate and the number of iterations per attacked image, respectively.

Federated Learning (FL)[1][2] emerged as an artificial intelligence training method that does not require sending data from peripheral devices(clients) to a central server. Rather, each client would download the central model from the server, train it over their private data, and send the resulting gradients of the private training back to the server, all of which are aggregated by a server-side algorithm to produce the next iteration of the central model. Ideally, mutually distrusted clients never communicate their private data, and yet they produce a central model that encompasses the entire clients' data. Extensive research is being conducted on optimizing the learning efficiency of FL on various aspects such as incentive mechanisms[3], communication speed[4], non-IID training[5], and client selection[6]. However, recent research reveals that sending the gradients of private training does not ensure complete data privacy, especially in a wide cross-device environment[7]. Moreover, as a federated system, FL has to protect itself against Byzantine Failure[8], Backdoor injection[9], Model Poisoning[10], and Data Poisoning[11]).

As the scale and size of the data increases significantly nowadays, federal learning (Bonawitz et al. [2019]) for high performance computing and machine learning has been much more important than ever before (Abadi et al. [2016]). People used to believe that sharing gradients seems to be safe to conceal the local training data during the training stage. However, Zhu et al. [2019] demonstrated that it was possible to recover raw data from the model training data by detecting gradients. They use generated random dummy data and minimise the distance between them and real data. Zhao et al. [2020] pushes the convergence algorithm even further. By replacing the original loss function with cross entropy loss, they achieve better fidelity threshold. In this paper, we propose using an additional dropout (Srivastava et al. [2014]) layer before feeding the data to the classifier. It is very effective in preventing leakage of raw data, as the training data cannot converge to a small RMSE even after 5,800 epochs with dropout rate set to 0.5.

Unlike traditional central training, federated learning (FL) improves the performance of the global model by sharing and aggregating local models rather than local data to protect the users' privacy. Although this training approach appears secure, some research has demonstrated that an attacker can still recover private data based on the shared gradient information. This on-the-fly reconstruction attack deserves to be studied in depth because it can occur at any stage of training, whether at the beginning or at the end of model training; no relevant dataset is required and no additional models need to be trained. We break through some unrealistic assumptions and limitations to apply this reconstruction attack in a broader range of scenarios. We propose methods that can reconstruct the training data from shared gradients or weights, corresponding to the FedSGD and FedAvg usage scenarios, respectively. We propose a zero-shot approach to restore labels even if there are duplicate labels in the batch. We study the relationship between the label and image restoration. We find that image restoration fails even if there is only one incorrectly inferred label in the batch; we also find that when batch images have the same label, the corresponding image is restored as a fusion of that class of images. Our approaches are evaluated on classic image benchmarks, including CIFAR-10 and ImageNet. The batch size, image quality, and the adaptability of the label distribution of our approach exceed those of GradInversion, the state-of-the-art.