We consider the problem of training generative models with deep neural networks as generators, i.e. to map latent codes to data points. Whereas the dominant paradigm combines simple priors over codes with complex deterministic models, we argue that it might be advantageous to use more flexible code distributions. We demonstrate how these distributions can be induced directly from the data. The benefits include: more powerful generative models, better modeling of latent structure and explicit control of the degree of generalization.

Deep generative models are powerful tools that have produced impressive results in recent years. These advances have been for the most part empirically driven, making it essential that we use high quality evaluation metrics. In this paper, we provide new insights into the Inception Score, a recently proposed and widely used evaluation metric for generative models, and demonstrate that it fails to provide useful guidance when comparing models. We discuss both suboptimalities of the metric itself and issues with its application. Finally, we call for researchers to be more systematic and careful when evaluating and comparing generative models, as the advancement of the field depends upon it.

This is a malware manipulation environment for OpenAI's gym. OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. This makes it possible to write agents that learn to manipulate PE files (e.g., malware) to achieve some objective (e.g., bypass AV) based on a reward provided by taking specific manipulation actions. Create an AI that learns through reinforcement learning which functionality-preserving transformations to make on a malware sample to break through / bypass machine learning static-analysis malware detection. There are two basic concepts in reinforcement learning: the environment (in our case, the malware sample) and the agent (namely, the algorithm used to change the environment). The agent sends actions to the environment, and the environment replies with observations and rewards (that is, a score).

Before an agent takes an action, it observes the communications from other agents from the previous time step as well as the locations of all entities and objects in the world. It stores that communication in a private recurrent neural network, giving it a memory for the words it hears. We use discrete communication actions (messages formed of separate, word-like symbols) sent over a differentiable communication channel. A communication channel is differentiable if it allows agents to directly inform each other about what message they should have sent at each time step, by slightly altering their messages to make a positive change in the reward both agents expect to receive. Agents accomplish this by calculating the gradient of future reward with respect to changes in the sent messages (i.e.

The development of model architectures and algorithms in the field of deep learning is largely constrained by the availability of efficient GPU implementations of elementary operations. One issue has been the lack of an efficient GPU implementation for sparse linear operations, which we're now releasing, together with initial results using them to implement a number of sparsity patterns. These initial results are promising but not definitive, and we invite the community to join us in pushing the limits of the architectures these kernels unlock. Dense layers (left) can be replaced with layers that are sparse and wide (center) or sparse and deep (right) while approximately retaining computation time. Sparse weight matrices, as opposed to dense weight matrices, have a large number of entries with a value of exactly zero.

Variational autoencoders (VAEs) learn representations of data by jointly training a probabilistic encoder and decoder network. Typically these models encode all features of the data into a single variable. Here we are interested in learning disentangled representations that encode distinct aspects of the data into separate variables. We propose to learn such representations using model architectures that generalise from standard VAEs, employing a general graphical model structure in the encoder and decoder. This allows us to train partially-specified models that make relatively strong assumptions about a subset of interpretable variables and rely on the flexibility of neural networks to learn representations for the remaining variables. We further define a general objective for semi-supervised learning in this model class, which can be approximated using an importance sampling procedure. We evaluate our framework's ability to learn disentangled representations, both by qualitative exploration of its generative capacity, and quantitative evaluation of its discriminative ability on a variety of models and datasets.

Speaker: Ilya Sutskever, OpenAI Hosted by: Vector Institute Date and Time: Thursday, December 14, 2017 - 12:00pm to 1:00pm Location: Fields Institute, Room 230 Abstract: In the first part, I will talk about meta learning, which is the problem of training a system that quickly learns to solve a wide variety of tasks. I will present meta reinforcement learning algorithms that can quickly solve simulated robotics tasks, and show how a simple meta learning approach can address the sim2real problem in robotics. The second part will be on self play. Self play systems provide a perfectly-fined grained curriculum, a potentially indefinite incentive for improvement, and a way of converting compute into data. I will present several recent results in self play and discuss their future potential.

Our latest robotics techniques allow robot controllers, trained entirely in simulation and deployed on physical robots, to react to unplanned changes in the environment as they solve simple tasks. That is, we've used these techniques to build closed-loop systems rather than open-loop ones as before. The simulator need not match the real-world in appearance or dynamics; instead, we randomize relevant aspects of the environment, from friction to action delays to sensor noise. Our new results provide more evidence that general-purpose robots can be built by training entirely in simulation, followed by a small amount of self-calibration in the real world. This robot was trained in simulation with dynamics randomization to push a puck to a goal.

It's been a few years since I was first exposed to reinforcement learning. What got me into it was seeing this video that had trained a neural network to play Mario. As someone who grew up playing Mario, seeing deep learning being applied to something I knew so well seemed to provided the perfect introduction to the topic. Sadly though, the project was written using Torch, and I was still a naive young programmer. I didn't get too far along before the frustrations with learning lua lead me to give up, and just focus on other projects instead.

AI is transforming every industry it touches from healthcare, to retail and advertising, finance, transport, education, agriculture and so many more. To take care of all the mundane tasks employees currently handle, freeing up their time to be more creative and perform the work that machines cannot do. Today, the rapidly advancing technology is used mostly by large enterprises through machine learning and predictive analytics. AI is not a technology of the future, it's happening now, and companies who fail to adopt it will get left behind. This paper will explore the application of AI in business with research contributions from leading minds in the field including Ankur Handa, Research Scientist, OpenAI, Ian Goodfellow, Senior Research Scientist, Google Brain.