To achieve a high learning accuracy, generative adversarial networks (GANs) must be fed by large datasets that adequately represent the data space. However, in many scenarios, the available datasets may be limited and distributed across multiple agents, each of which is seeking to learn the distribution of the data on its own. In such scenarios, the local datasets are inherently private and agents often do not wish to share them. In this paper, to address this multi-agent GAN problem, a novel brainstorming GAN (BGAN) architecture is proposed using which multiple agents can generate real-like data samples while operating in a fully distributed manner and preserving their data privacy. BGAN allows the agents to gain information from other agents without sharing their real datasets but by "brainstorming" via the sharing of their generated data samples. Therefore, the proposed BGAN yields a higher accuracy compared with a standalone GAN model and its architecture is fully distributed and does not need any centralized controller. Moreover, BGANs are shown to be scalable and not dependent on the hyperparameters of the agents' deep neural networks (DNNs) thus enabling the agents to have different DNN architectures. Theoretically, the interactions between BGAN agents are analyzed as a game whose unique Nash equilibrium is derived. Experimental results show that BGAN can generate real-like data samples with higher quality compared to other distributed GAN architectures.

Item Response Theory is a ubiquitous algorithm used around the world to understand humans based on their responses to questions in fields as diverse as education, medicine and psychology. However, for medium to large datasets, contemporary solutions pose a tradeoff: either have bayesian, interpretable, accurate estimates or have fast computation. We introduce variational inference and deep generative models to Item Response Theory to offer the best of both worlds. The resulting algorithm is (a) orders of magnitude faster when inferring on the classical model, (b) naturally extends to more complicated input than binary correct/incorrect, and more expressive deep bayesian models of responses. Applying this method to five large-scale item response datasets from cognitive science and education, we find improvements in imputing missing data and better log likelihoods. The open-source algorithm is immediately usable.

We develop a corrective mechanism for neural network approximation: the total available non-linear units are divided into multiple groups and the first group approximates the function under consideration, the second group approximates the error in approximation produced by the first group and corrects it, the third group approximates the error produced by the first and second groups together and so on. This technique yields several new representation and learning results for neural networks. First, we show that two-layer neural networks in the random features regime (RF) can memorize arbitrary labels for arbitrary points under under Euclidean distance separation condition using $\tilde{O}(n)$ ReLU or Step activation functions which is optimal in $n$ up to logarithmic factors. Next, we give a powerful representation result for two-layer neural networks with ReLU and smoothed ReLU units which can achieve a squared error of at most $\epsilon$ with $O(C(a,d)\epsilon^{-1/(a+1)})$ for $a \in \mathbb{N}\cup\{0\}$ when the function is smooth enough (roughly when it has $\Theta(ad)$ bounded derivatives). In certain cases $d$ can be replaced with effective dimension $q \ll d$. Previous results of this type implement Taylor series approximation using deep architectures. We also consider three-layer neural networks and show that the corrective mechanism yields faster representation rates for smooth radial functions. Lastly, we obtain the first $O(\mathrm{subpoly}(1/\epsilon))$ upper bound on the number of neurons required for a two layer network to learn low degree polynomials up to squared error $\epsilon$ via gradient descent. Even though deep networks can express these polynomials with $O(\mathrm{polylog}(1/\epsilon))$ neurons, the best learning bounds on this problem require $\mathrm{poly}(1/\epsilon)$ neurons.

We propose a method for generating an electrocardiogram (ECG) signal for one cardiac cycle using a variational autoencoder. Using this method we extracted a vector of new 25 features, which in many cases can be interpreted. The generated ECG has quite natural appearance. The low value of the Maximum Mean Discrepancy metric, 0.00383, indicates good quality of ECG generation too. The extracted new features will help to improve the quality of automatic diagnostics of cardiovascular diseases. Also, generating new synthetic ECGs will allow us to solve the issue of the lack of labeled ECG for use them in supervised learning.

Federated learning systems are vulnerable to attacks from malicious clients. As the central server in the system cannot govern the behaviors of the clients, a rogue client may initiate an attack by sending malicious model updates to the server, so as to degrade the learning performance or enforce targeted model poisoning attacks (a.k.a. Therefore, timely detecting these malicious model updates and the underlying attackers becomes critically important. In this work, we propose a new framework for robust federated learning where the central server learns to detect and remove the malicious model updates using a powerful detection model, leading to targeted defense . We evaluate our solution in both image classification and sentiment analysis tasks with a variety of machine learning models. Experimental results show that our solution ensures robust federated learning that is resilient to both the Byzantine attacks and the targeted model poisoning attacks. 1 Introduction Federated learning (FL) comes as a new distributed machine learning (ML) paradigm where multiple clients (e.g., mobile devices) collaboratively train an ML model without revealing their private data [ McMahan et al., 2017; Y ang et al., 2019b; Kairouz et al., 2019 ] . In a typical FL setting, a central server is used to maintain a global model and coordinate the clients. Each client transfers the local model updates to the central server for immediate aggregation, while keeping the raw data in their local storage.

The newly released book "The A.I. Age" represents a historic moment and new landmark in the story of AI with its February 2020 publication. The front cover of the book has been entirely created by Artificial Intelligence (AI) by selecting the winning cover design and overlaying the text. AI machines can't be creative? A specific type of machine learning called Generative Adversarial Networks (GANs), can create art. Author Adam Riccoboni, CEO of Critical Future, a leading AI firm, used this cutting-edge AI to create his book cover's design while writing The AI Age. The process involved feeding 200,000 book covers into the deep learning system.

Companies need to align the AI strategy with the business strategy. First the company needs a business strategy. Can the AI help with that? AI is being applied to the model decision-making of governments and corporations. A recent article in Forbes described the Real Time Strategy (RTS) technology involved in Google DeepMind's gaming software that works with "imperfect information."

Trust and transparency are at the forefront of conversations related to artificial intelligence(AI) these days. While we intuitively understand the idea of trusting AI agents, we are still trying to figure out the specific mechanics to translate trust and transparency into programmatic constructs. After all, what does trust means in the context of an AI system? Trust is a foundational building block of human socio-economic dynamics. In software development, during the last few decades, we steadily built mechanisms for asserting trust on specific applications.

Adrian Schwaiger, research engineer at the Fraunhofer Institute for Cognitive Systems IKS, will be presenting his new paper »Benchmarking Uncertainty Estimation Methods for Deep Learning With Safety-Related Metrics« at the SafeAI workshop. The workshop belongs to the AAAI-20 Conference in New York City. In this paper, Adrian Schwaiger and his colleagues Maximilian Henne, Karsten Roscher and Gereon Weiss compare current models of measuring uncertainty in deep neural networks in order to optimally combine the safety and performance of cognitive systems. Deep neural networks generally perform very well in giving accurate predictions, but they often fail in recognizing when these predictions may be wrong. This absence of awareness regarding the reliability of given outputs is a big obstacle in deploying such models in safety-critical applications for example in autonomous vehicles or in medical technology.

In the last few years AI researchers have had increasing success in speeding up neural network training through data-parallelism, which splits large batches of data across many machines. Researchers have successfully used batch sizes of tens of thousands for image classification and language modeling, and even millions for RL agents that play the game Dota 2. These large batches allow increasing amounts of compute to be efficiently poured into the training of a single model, and are an important enabler of the fast growth in AI training compute. However, batch sizes that are too large show rapidly diminishing algorithmic returns, and it's not clear why these limits are larger for some tasks and smaller for others.[1] We have found that by measuring the gradient noise scale, a simple statistic that quantifies the signal-to-noise ratio of the network gradients,[2] we can approximately predict the maximum useful batch size. Heuristically, the noise scale measures the variation in the data as seen by the model (at a given stage in training).