Sparse tensors appear frequently in federated deep learning, either as a direct artifact of the deep neural network's gradients, or as a result of an explicit sparsification process. Existing communication primitives are agnostic to the peculiarities of deep learning; consequently, they impose unnecessary communication overhead. This paper introduces DeepReduce, a versatile framework for the compressed communication of sparse tensors, tailored to federated deep learning. DeepReduce decomposes sparse tensors into two sets, values and indices, and allows both independent and combined compression of these sets. We support a variety of common compressors, such as Deflate for values, or run-length encoding for indices. We also propose two novel compression schemes that achieve superior results: curve fitting-based for values, and bloom filter-based for indices. DeepReduce is orthogonal to existing gradient sparsifiers and can be applied in conjunction with them, transparently to the end-user, to significantly lower the communication overhead. As proof of concept, we implement our approach on TensorFlow and PyTorch. Our experiments with large real models demonstrate that DeepReduce transmits 320% less data than existing sparsifiers, without affecting accuracy.

Sparse tensors appear frequently in federated deep learning, either as a direct artifact of the deep neural network's gradients, or as a result of an explicit sparsification process. Existing communication primitives are agnostic to the peculiarities of deep learning; consequently, they impose unnecessary communication overhead. This paper introduces DeepReduce, a versatile framework for the compressed communication of sparse tensors, tailored to federated deep learning. DeepReduce decomposes sparse tensors into two sets, values and indices, and allows both independent and combined compression of these sets. We support a variety of common compressors, such as Deflate for values, or run-length encoding for indices.

Batch Normalization (BN) is commonly used in modern deep learning to improve stability and speed up convergence in centralized training. In federated learning (FL) with non-IID decentralized data, previous works observed that training with BN could hinder performance due to the mismatch of the BN statistics between training and testing. Group Normalization (GN) is thus more often used in FL as an alternative to BN. In this paper, we identify a more fundamental issue of BN in FL that makes BN inferior even with high-frequency communication between clients and servers. We then propose a frustratingly simple treatment, which significantly improves BN and makes it outperform GN across a wide range of FL settings. Along with this study, we also reveal an unreasonable behavior of BN in FL. We find it quite robust in the low-frequency communication regime where FL is commonly believed to degrade drastically. We hope that our study could serve as a valuable reference for future practical usage and theoretical analysis in FL.

The vast increase of Internet of Things (IoT) technologies and the ever-evolving attack vectors have increased cyber-security risks dramatically. A common approach to implementing AI-based Intrusion Detection systems (IDSs) in distributed IoT systems is in a centralised manner. However, this approach may violate data privacy and prohibit IDS scalability. Therefore, intrusion detection solutions in IoT ecosystems need to move towards a decentralised direction. Federated Learning (FL) has attracted significant interest in recent years due to its ability to perform collaborative learning while preserving data confidentiality and locality. Nevertheless, most FL-based IDS for IoT systems are designed under unrealistic data distribution conditions. To that end, we design an experiment representative of the real world and evaluate the performance of an FL-based IDS. For our experiments, we rely on TON-IoT, a realistic IoT network traffic dataset, associating each IP address with a single FL client. Additionally, we explore pre-training and investigate various aggregation methods to mitigate the impact of data heterogeneity. Lastly, we benchmark our approach against a centralised solution. The comparison shows that the heterogeneous nature of the data has a considerable negative impact on the model's performance when trained in a distributed manner. However, in the case of a pre-trained initial global FL model, we demonstrate a performance improvement of over 20% (F1-score) compared to a randomly initiated global model.

It is well know that Artificial Intelligence (AI) identifies in a broad sense the ability of a machine to learn from experience, to simulate the human intelligence, to adapt to new scenarios, and to get engaged in human-like activities. AI identifies an interdisciplinary area which includes computer science, robotics, engineering, mathematics. Over the years, it has made a rapid progress: it will contribute to the society transformation through the adoption of innovating technologies and creative intelligence and the large-scale implementation of AI in technologies such as IoT, smart speakers, chat-bots, cybersecurity, 3D printing, drones, face emotions analysis, sentiment analysis, natural language processing, and their applications to human resources, marketing, finance, and many others. With the term Machine learning (ML), instead, we identify a branch of AI in which algorithms are used to learn from data to make future decisions or predictions. ML algorithms are trained on past data in order to make future predictions or to support the decision making process. Deep Learning (DL), instead, is a subset of ML and it includes a large family of ML methods and architectures based on Artificial Neural Networks (ANNs). It includes Deep Neural Networks, Deep Belief Networks, Deep Reinforcement Learning, Recurrent Neural Networks and Convolutional Neural Networks, to mention a few of them. DL algorithms have been used in several applications including computer vision, speech recognition, natural language processing, bioinformatics, medical image analysis, and in most of these areas they have demonstrated to perform better than humans. In the recent years DL has disrupted every application domain and it provides a robust, generalized, and scalable approach to work with different data types, including time-series data [1-4].

Let's start with the one minute version: I was part of the EF12 London cohort in 2019, where I met my co-founder. A privacy-preserving medical-data marketplace and AI platform built around federated deep learning. The purpose of the platform would have been to allow data scientists to train deep learning models on highly sensitive healthcare data without that data ever leaving the hospitals. At the same time, thanks to a novel data monetization strategy and marketplace component, hospitals would have been empowered to make money from the data they are generating. We received pre-seed funding, valued at $1 million. Then the race for demo day began with frantic product building and non-stop business development.

Smart doorbells have been playing an important role in protecting Furthermore, the processing and storage of multiple video streams our modern homes. Existing approaches of sending video streams make the subscription more costly. Secondly, this design requires to a centralized server (or Cloud) for video analytics have been a huge amount of reliable bandwidth, which may not always be facing many challenges such as latency, bandwidth cost and more had. Third, even if we assume that we could address latency and importantly users' privacy concerns. To address these challenges, bandwidth issue by empowering a sophisticated infrastructure, a this paper showcases the ability of an intelligent smart doorbell large class of video-based applications may not be suitable because based on Federated Deep Learning, which can deploy and manage of regulations and security concerns of sharing data as there is an video analytics applications such as a smart doorbell across Edge involvement of biometric data of residents.

When used well, deep learning technology can boost enterprises looking to collect, analyze and interpret big data. Successful use cases for deep learning vary from natural language processing to medical diagnosis automation, but all require big data analytics. For a deep learning investment to be deployed effectively, enterprises need to first accurately train the models despite bias and data challenges. Through proper data gathering, new data approaches, reinforcement learning, strong workflows and federated deep learning, companies can properly tackle the challenges of deep learning model training. These five tips can help guide an enterprise into training deep models the right way.

Federated Learning (FL) has been proved to be an effective learning framework when data cannot be centralized due to privacy, communication costs, and regulatory restrictions. When training deep learning models under an FL setting, people employ the predefined model architecture discovered in the centralized environment. However, this predefined architecture may not be the optimal choice because it may not fit data with non-identical and independent distribution (non-IID). Thus, we advocate automating federated learning (AutoFL) to improve model accuracy and reduce the manual design effort. We specifically study AutoFL via Neural Architecture Search (NAS), which can automate the design process. We propose a Federated NAS (FedNAS) algorithm to help scattered workers collaboratively searching for a better architecture with higher accuracy. We also build a system based on FedNAS. Our experiments on non-IID dataset show that the architecture searched by FedNAS can outperform the manually predefined architecture.