OpenAI, a $1 billion (£687 million) artificial intelligence company backed by Elon Musk, has built a "gym" where developers can train their AI systems to get smarter. Using OpenAI's open source toolkit, available for download now, developers can access "environments" where they can test their AI bots. The OpenAI Gym, currently in beta, provides a number of environments, including more than 50 Atari games, such as "Space Invaders," "Pong," "Asteroids" and "Pac-Man". Developers can also test their AIs on board games like Go, which was recently mastered by an agent built by London startup Google DeepMind. "Over time, we plan to greatly expand this collection of environments," wrote OpenAI's Greg Brockman and John Schulman in a blog post.

We propose a probabilistic model for interpreting gene expression levels that are observed through single-cell RNA sequencing. In the model, each cell has a low-dimensional latent representation. Additional latent variables account for technical effects that may erroneously set some observations of gene expression levels to zero. Conditional distributions are specified by neural networks, giving the proposed model enough flexibility to fit the data well. We use variational inference and stochastic optimization to approximate the posterior distribution. The inference procedure scales to over one million cells, whereas competing algorithms do not. Even for smaller datasets, for several tasks, the proposed procedure outperforms state-of-the-art methods like ZIFA and ZINB-WaVE. We also extend our framework to account for batch effects and other confounding factors, and propose a Bayesian hypothesis test for differential expression that outperforms DESeq2.

I'm stoked!! I'm broadcasting live on Instagram Stories, so be sure to stay along for the ride by following and leaving a comment for me!"

We consider the problem of training generative models with deep neural networks as generators, i.e. to map latent codes to data points. Whereas the dominant paradigm combines simple priors over codes with complex deterministic models, we argue that it might be advantageous to use more flexible code distributions. We demonstrate how these distributions can be induced directly from the data. The benefits include: more powerful generative models, better modeling of latent structure and explicit control of the degree of generalization.

Deep generative models are powerful tools that have produced impressive results in recent years. These advances have been for the most part empirically driven, making it essential that we use high quality evaluation metrics. In this paper, we provide new insights into the Inception Score, a recently proposed and widely used evaluation metric for generative models, and demonstrate that it fails to provide useful guidance when comparing models. We discuss both suboptimalities of the metric itself and issues with its application. Finally, we call for researchers to be more systematic and careful when evaluating and comparing generative models, as the advancement of the field depends upon it.

This is a malware manipulation environment for OpenAI's gym. OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. This makes it possible to write agents that learn to manipulate PE files (e.g., malware) to achieve some objective (e.g., bypass AV) based on a reward provided by taking specific manipulation actions. Create an AI that learns through reinforcement learning which functionality-preserving transformations to make on a malware sample to break through / bypass machine learning static-analysis malware detection. There are two basic concepts in reinforcement learning: the environment (in our case, the malware sample) and the agent (namely, the algorithm used to change the environment). The agent sends actions to the environment, and the environment replies with observations and rewards (that is, a score).

Before an agent takes an action, it observes the communications from other agents from the previous time step as well as the locations of all entities and objects in the world. It stores that communication in a private recurrent neural network, giving it a memory for the words it hears. We use discrete communication actions (messages formed of separate, word-like symbols) sent over a differentiable communication channel. A communication channel is differentiable if it allows agents to directly inform each other about what message they should have sent at each time step, by slightly altering their messages to make a positive change in the reward both agents expect to receive. Agents accomplish this by calculating the gradient of future reward with respect to changes in the sent messages (i.e.

The development of model architectures and algorithms in the field of deep learning is largely constrained by the availability of efficient GPU implementations of elementary operations. One issue has been the lack of an efficient GPU implementation for sparse linear operations, which we're now releasing, together with initial results using them to implement a number of sparsity patterns. These initial results are promising but not definitive, and we invite the community to join us in pushing the limits of the architectures these kernels unlock. Dense layers (left) can be replaced with layers that are sparse and wide (center) or sparse and deep (right) while approximately retaining computation time. Sparse weight matrices, as opposed to dense weight matrices, have a large number of entries with a value of exactly zero.

Variational autoencoders (VAEs) learn representations of data by jointly training a probabilistic encoder and decoder network. Typically these models encode all features of the data into a single variable. Here we are interested in learning disentangled representations that encode distinct aspects of the data into separate variables. We propose to learn such representations using model architectures that generalise from standard VAEs, employing a general graphical model structure in the encoder and decoder. This allows us to train partially-specified models that make relatively strong assumptions about a subset of interpretable variables and rely on the flexibility of neural networks to learn representations for the remaining variables. We further define a general objective for semi-supervised learning in this model class, which can be approximated using an importance sampling procedure. We evaluate our framework's ability to learn disentangled representations, both by qualitative exploration of its generative capacity, and quantitative evaluation of its discriminative ability on a variety of models and datasets.

Speaker: Ilya Sutskever, OpenAI Hosted by: Vector Institute Date and Time: Thursday, December 14, 2017 - 12:00pm to 1:00pm Location: Fields Institute, Room 230 Abstract: In the first part, I will talk about meta learning, which is the problem of training a system that quickly learns to solve a wide variety of tasks. I will present meta reinforcement learning algorithms that can quickly solve simulated robotics tasks, and show how a simple meta learning approach can address the sim2real problem in robotics. The second part will be on self play. Self play systems provide a perfectly-fined grained curriculum, a potentially indefinite incentive for improvement, and a way of converting compute into data. I will present several recent results in self play and discuss their future potential.