Collaborating Authors

model update

What is Federated Learning?


There is also a paper that describes a scalable production system for FL for mobile devices, "Towards Federated Learning at Scale: System Design" (Mar.

User Scheduling for Federated Learning Through Over-the-Air Computation Artificial Intelligence

A new machine learning (ML) technique termed as federated learning (FL) aims to preserve data at the edge devices and to only exchange ML model parameters in the learning process. FL not only reduces the communication needs but also helps to protect the local privacy. Although FL has these advantages, it can still experience large communication latency when there are massive edge devices connected to the central parameter server (PS) and/or millions of model parameters involved in the learning process. Over-the-air computation (AirComp) is capable of computing while transmitting data by allowing multiple devices to send data simultaneously by using analog modulation. To achieve good performance in FL through AirComp, user scheduling plays a critical role. In this paper, we investigate and compare different user scheduling policies, which are based on various criteria such as wireless channel conditions and the significance of model updates. Receiver beamforming is applied to minimize the mean-square-error (MSE) of the distortion of function aggregation result via AirComp. Simulation results show that scheduling based on the significance of model updates has smaller fluctuations in the training process while scheduling based on channel condition has the advantage on energy efficiency.

A Payload Optimization Method for Federated Recommender Systems Artificial Intelligence

Federated Learning (FL) McMahan et al. [2017], a privacy-by-design machine learning approach, has introduced new ways to build recommender systems (RS). Unlike traditional approaches, the FL approach means that there is no longer a need to collect and store the users' private data on central servers, while making it possible to train robust recommendation models. In practice, FL distributes the model training process to the users' devices (i.e., the client or edge devices), thus allowing a global model to be trained using the user-specific local models. Each user updates the global model locally using their personal data and sends the local model updates to a server that aggregates them according to a pre-defined scheme. This is in order to update the global model. A prominent direction of research in this domain is based on Federated Collaborative Filtering (FCF) Ammad-Ud-Din et al. [2019], Chai et al. [2020], Dolui et al. [2019] that extends the standard Collaborative Filtering (CF) Hu et al. [2008] model to the federated mode. CF is one of the most frequently used matrix factorization models used to generate personalized recommendations either independently or in combination with other types of model Koren et al. [2009].

Regression during model updates


Consider you have a prediction system h1 (example a photo tagger) whose output is consumed in real world (example tagging your photos on phone). Now, you train a system h2 whose aggregate metrics suggest that it is better than h1. Let's consider an unlabeled dataset D of examples (a pool of all user photos). Prediction update refers to the process where h2 is used to score examples in dataset D and update the predictions provided by h1. The problem here is that even though h2 is better than h1 globally, we haven't determined if h2 is significantly worse for some users or some specific pattern of examples.

Privacy Assessment of Federated Learning using Private Personalized Layers Artificial Intelligence

Federated Learning (FL) is a collaborative scheme to train a learning model across multiple participants without sharing data. While FL is a clear step forward towards enforcing users' privacy, different inference attacks have been developed. In this paper, we quantify the utility and privacy trade-off of a FL scheme using private personalized layers. While this scheme has been proposed as local adaptation to improve the accuracy of the model through local personalization, it has also the advantage to minimize the information about the model exchanged with the server. However, the privacy of such a scheme has never been quantified. Our evaluations using motion sensor dataset show that personalized layers speed up the convergence of the model and slightly improve the accuracy for all users compared to a standard FL scheme while better preventing both attribute and membership inferences compared to a FL scheme using local differential privacy.

Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix Artificial Intelligence

We show that aggregated model updates in federated learning may be insecure. An untrusted central server may disaggregate user updates from sums of updates across participants given repeated observations, enabling the server to recover privileged information about individual users' private training data via traditional gradient inference attacks. Our method revolves around reconstructing participant information (e.g: which rounds of training users participated in) from aggregated model updates by leveraging summary information from device analytics commonly used to monitor, debug, and manage federated learning systems. Our attack is parallelizable and we successfully disaggregate user updates on settings with up to thousands of participants. We quantitatively and qualitatively demonstrate significant improvements in the capability of various inference attacks on the disaggregated updates. Our attack enables the attribution of learned properties to individual users, violating anonymity, and shows that a determined central server may undermine the secure aggregation protocol to break individual users' data privacy in federated learning.

A Privacy-Preserving and Trustable Multi-agent Learning Framework Artificial Intelligence

Distributed multi-agent learning enables agents to cooperatively train a model without requiring to share their datasets. While this setting ensures some level of privacy, it has been shown that, even when data is not directly shared, the training process is vulnerable to privacy attacks including data reconstruction and model inversion attacks. Additionally, malicious agents that train on inverted labels or random data, may arbitrarily weaken the accuracy of the global model. This paper addresses these challenges and presents Privacy-preserving and trustable Distributed Learning (PT-DL), a fully decentralized framework that relies on Differential Privacy to guarantee strong privacy protections of the agents' data, and Ethereum smart contracts to ensure trustability. The paper shows that PT-DL is resilient up to a 50% collusion attack, with high probability, in a malicious trust model and the experimental evaluation illustrates the benefits of the proposed model as a privacy-preserving and trustable distributed multi-agent learning system on several classification tasks.

Latency Analysis of Consortium Blockchained Federated Learning Machine Learning

A decentralized federated learning architecture is proposed to apply to the Businesses-to-Businesses scenarios by introducing the consortium blockchain in this paper. We introduce a model verification mechanism to ensure the quality of local models trained by participators. To analyze the latency of the system, a latency model is constructed by considering the work flow of the architecture. Finally the experiment results show that our latency model does well in quantifying the actual delays.

Pre-trained deep learning models update (February 2021)


Today was a fun and exciting day at the Esri Federal GIS Conference 2021 highlighted by great user presentations, inspiring talks, and a powerful technology showcase. The imagery and remote sensing demonstration showed how AI was effectively put to use in a SAAS environment. Driving the AI was a pre-trained model that is downloadable for all users from ArcGIS Living Atlas. This is just one of the many models that have been released on ArcGIS Living Atlas of the World. Ever since the pre-trained geospatial deep learning models were released on ArcGIS Living Atlas, they have been well received.

Untargeted Poisoning Attack Detection in Federated Learning via Behavior Attestation Artificial Intelligence

Federated Learning (FL) is a paradigm in Machine Learning (ML) that addresses data privacy, security, access rights and access to heterogeneous information issues by training a global model using distributed nodes. Despite its advantages, there is an increased potential for cyberattacks on FL-based ML techniques that can undermine the benefits. Model-poisoning attacks on FL target the availability of the model. The adversarial objective is to disrupt the training. We propose attestedFL, a defense mechanism that monitors the training of individual nodes through state persistence in order to detect a malicious worker. A fine-grained assessment of the history of the worker permits the evaluation of its behavior in time and results in innovative detection strategies. We present three lines of defense that aim at assessing if the worker is reliable by observing if the node is really training, advancing towards a goal. Our defense exposes an attacker's malicious behavior and removes unreliable nodes from the aggregation process so that the FL process converge faster. Through extensive evaluations and against various adversarial settings, attestedFL increased the accuracy of the model between 12% to 58% under different scenarios such as attacks performed at different stages of convergence, attackers colluding and continuous attacks.