The third way to get involved with OpenAI is as an OpenAI Scholar. Under this program OpenAI is providing 6-10 stipends and mentorship to individuals from underrepresented groups to study deep learning full-time for 3 months and open-source a project. This is a remote program and is open to anyone with US work authorization located in US timezones. In return, scholars are asked to document their experiences of studying deep learning and hopefully inspire others to do the same.

We present an application of deep generative models in the context of partial-differential equation (PDE) constrained inverse problems. We combine a generative adversarial network (GAN) representing an a priori model that creates subsurface geological structures and their petrophysical properties, with the numerical solution of the PDE governing the propagation of acoustic waves within the earth's interior. We perform Bayesian inversion using an approximate Metropolis-adjusted Langevin algorithm (MALA) to sample from the posterior given seismic observations. Gradients with respect to the model parameters governing the forward problem are obtained by solving the adjoint of the acoustic wave equation. Gradients of the mismatch with respect to the latent variables are obtained by leveraging the differentiable nature of the deep neural network used to represent the generative model. We show that approximate MALA sampling allows efficient Bayesian inversion of model parameters obtained from a prior represented by a deep generative model, obtaining a diverse set of realizations that reflect the observed seismic response.

I recently read the "Learning to generate reviews and discovering sentiment" paper by OPENAI and found it to be super cool. But I could not understand how they are using the language model as feature extractor. Suppose we have 150 characters in a review, how do we extract features from these 150 characters when our input is 64 characters at a time.

Deep generative models can learn to generate realistic-looking images on several natural image datasets, but many of the most effective methods are adversarial methods, which require careful balancing of training between a generator network and a discriminator network. Maximum mean discrepancy networks (MMD-nets) avoid this issue using the kernel trick, but unfortunately they have not on their own been able to match the performance of adversarial training. We present a new method of training MMD-nets, based on learning a mapping of samples from the data and from the model into a lower dimensional space, in which MMD training can be more effective. We call these networks ratio matching MMD networks (RM-MMDnets). We train the mapping to preserve density ratios between the densities over the low-dimensional space and the original space. This ensures that matching the model distribution to the data in the low-dimensional space will also match the original distributions. We show that RM-MMDnets have better performance and better stability than recent adversarial methods for training MMD-nets.

We propose and study the problem of distribution-preserving lossy compression. Motivated by the recent advances in extreme image compression which allow to maintain artifact-free reconstructions even at very low bitrates, we propose to optimize the rate-distortion tradeoff under the constraint that the reconstructed samples follow the distribution of the training data. Such a compression system recovers both ends of the spectrum: On one hand, at zero bitrate it learns a generative model of the data, and at high enough bitrates it achieves perfect reconstruction. Furthermore, for intermediate bitrates it smoothly interpolates between matching the distribution of the training data and perfectly reconstructing the training samples. We study several methods to approximately solve the proposed optimization problem, including a novel combination of Wasserstein GAN and Wasserstein Autoencoder, and present strong theoretical and empirical results for the proposed compression system.

We introduce a new approach to learning in hierarchical latent-variable generative models called the "distributed distributional code Helmholtz machine", which emphasises flexibility and accuracy in the inferential process. In common with the original Helmholtz machine and later variational autoencoder algorithms (but unlike adverserial methods) our approach learns an explicit inference or "recognition" model to approximate the posterior distribution over the latent variables. Unlike in these earlier methods, the posterior representation is not limited to a narrow tractable parameterised form (nor is it represented by samples). To train the generative and recognition models we develop an extended wake-sleep algorithm inspired by the original Helmholtz Machine. This makes it possible to learn hierarchical latent models with both discrete and continuous variables, where an accurate posterior representation is essential. We demonstrate that the new algorithm outperforms current state-of-the-art methods on synthetic, natural image patch and the MNIST data sets.

The process of translation is ambiguous, in that there are typically many valid trans- lations for a given sentence. This gives rise to significant variation in parallel cor- pora, however, most current models of machine translation do not account for this variation, instead treating the prob- lem as a deterministic process. To this end, we present a deep generative model of machine translation which incorporates a chain of latent variables, in order to ac- count for local lexical and syntactic varia- tion in parallel corpora. We provide an in- depth analysis of the pitfalls encountered in variational inference for training deep generative models. Experiments on sev- eral different language pairs demonstrate that the model consistently improves over strong baselines.

Artificial intelligence research organization OpenAI recently released a report that shows the amount of compute power needed for training runs in the largest machine learning systems has increased by 300,000 times since 2012. Because machine learning results improve when given additional computing resources, we'll likely see even greater demands for silicon infrastructure to drive better results. Enterprises are increasingly using machine learning to automate complex problems and analytical tasks. But OpenAI's research shows there's a key challenge ahead: How can enterprises build the infrastructure they need to produce the business results they want when the technical requirements keep changing? First off, enterprises should try to find the least complicated algorithm necessary to solve the business problem at hand.

This is a genuine question! If you look into the code, you'll find they are calling properties on the observation space variables that are passed into the learners that don't exist. I am trying to do policysearch with a dict based observationspace. Nothing suggests that wouldn't be possible. None, None) # None for shape and dtype, since it'll require special handling so ... rewriting the code to be a tuple now.

Demand for compute to train artificial intelligence models has shot up enormously over the past six years and is showing no signs of slowing down. Not for profit research firm OpenAI - which is sponsored by Peter Thiel, Elon Musk, Microsoft and Amazon Web Services, among others - published an analysis that showed the amount of compute used for the largest AI training runs has doubled every three-and-a-half months since 2012. This means compute amounts have grown by more than 300,000 times over the past six years, OpenAI said. In comparison, the well-known Moore's Law, which observed the number of transistors in an integrated circuit would double every year-and-a-half, would yield only a twelve-fold increase in performance over the same period. Part of the reason AI models still have enough compute is because of the use of massively parallel video cards or graphics processing units (GPUs) that can have thousands of cores per unit. Furthermore, over the past two years, optimisations such as huge batch sizes, architecture search and expert iteration using improved and specialised hardware such as Tensor processing units (TPUs) and fast data interconnects have increased past limits for algorithmic parallelism.