Recent work has shown that deep generative models can assign higher likelihood to out-of-distribution data sets than to their training data. We posit that this phenomenon is caused by a mismatch between the model's typical set and its areas of high probability density. In-distribution inputs should reside in the former but not necessarily in the latter, as previous work has presumed. To determine whether or not inputs reside in the typical set, we propose a statistically principled, easy-to-implement test using the empirical distribution of model likelihoods. The test is model agnostic and widely applicable, only requiring that the likelihood can be computed or closely approximated. We report experiments showing that our procedure can successfully detect the out-of-distribution sets in several of the challenging cases reported by Nalisnick et al. (2019).

Discriminative neural networks offer little or no performance guarantees when deployed on data not generated by the same process as the training distribution. On such out-of-distribution (OOD) inputs, the prediction may not only be erroneous, but confidently so, limiting the safe deployment of classifiers in real-world applications. One such challenging application is bacteria identification based on genomic sequences, which holds the promise of early detection of diseases, but requires a model that can output low confidence predictions on OOD genomic sequences from new bacteria that were not present in the training data. We introduce a genomics dataset for OOD detection that allows other researchers to benchmark progress on this important problem. We investigate deep generative model based approaches for OOD detection and observe that the likelihood score is heavily affected by population level background statistics. We propose a likelihood ratio method for deep generative models which effectively corrects for these confounding background statistics. We benchmark the OOD detection performance of the proposed method against existing approaches on the genomics dataset and show that our method achieves state-of-the-art performance. We demonstrate the generality of the proposed method by showing that it significantly improves OOD detection when applied to deep generative models of images.

What you will learn in this course? In this course, you'll work with more complex environments, specifically provided by the OpenAI Gym: CartPole Mountain Car Atari games to train effective learning agents so you'll need new techniques. We've seen that reinforcement learning is an entirely different kind of machine learning than supervised and unsupervised learning.Supervised and unsupervised machine learning algorithms are for making predictions about data and analyzing, while reinforcement learning is about training an agent to interact with an environment and maximize its reward. Deep reinforcement learning and AI has a lot of potentials also carries huge risk. One main principle of training reinforcement learning agents is that there are unintended consequences when training an AI.

Reliable training of generative adversarial networks (GANs) typically require massive datasets in order to model complicated distributions. However, in several applications, training samples obey invariances that are \textit{a priori} known; for example, in complex physics simulations, the training data obey universal laws encoded as well-defined mathematical equations. In this paper, we propose a new generative modeling approach, InvNet, that can efficiently model data spaces with known invariances. We devise an adversarial training algorithm to encode them into data distribution. We validate our framework in three experimental settings: generating images with fixed motifs; solving nonlinear partial differential equations (PDEs); and reconstructing two-phase microstructures with desired statistical properties. We complement our experiments with several theoretical results.

Sounds a lot like the road that led to an AI winter historically... I think we're well past the point where that's a genuine risk of AI interest globally cooling down at all (it's already very practical and profitable in many arenas just with what we have) but openAI themselves? If historical trends are any indication, that kind of talk will buy them at most 5 years of normal investor questions, 5 years of severe questions, then bankruptcy. They've very generously got a decade to figure something actually practical out, and realistically the clock might only have five years on it or less. Wonder if they'll invent a thing that'll teach them to make money before then, haha.

State-of-the-art deep learning methods have shown a remarkable capacity to model complex data domains, but struggle with geospatial data. In this paper, we introduce SpaceGAN, a novel generative model for geospatial domains that learns neighbourhood structures through spatial conditioning. We propose to enhance spatial representation beyond mere spatial coordinates, by conditioning each data point on feature vectors of its spatial neighbours, thus allowing for a more flexible representation of the spatial structure. To overcome issues of training convergence, we employ a metric capturing the loss in local spatial autocorrelation between real and generated data as stopping criterion for SpaceGAN parametrization. This way, we ensure that the generator produces synthetic samples faithful to the spatial patterns observed in the input. SpaceGAN is successfully applied for data augmentation and outperforms compared to other methods of synthetic spatial data generation. Finally, we propose an ensemble learning framework for the geospatial domain, taking augmented SpaceGAN samples as training data for a set of ensemble learners. We empirically show the superiority of this approach over conventional ensemble learning approaches and rivaling spatial data augmentation methods, using synthetic and real-world prediction tasks. Our findings suggest that SpaceGAN can be used as a tool for (1) artificially inflating sparse geospatial data and (2) improving generalization of geospatial models.

Every environment is fixed to "CartPole-v1" from OpenAI GYM. You can just focus on the implementations.

Competition between humans and artificial intelligence (A.I.) usually plays out in research papers. Occasionally, there's a public performance of a game of chess or Go in front of a staid crowd. Last month in Vancouver, British Columbia, however, I saw something entirely different. The Canadian city was playing host to The International, an annual tournament for the video game Dota 2 boasting a $25 million prize pool -- the largest in esports history. The event was raucous, tribal.

Modelling biologically-plausible neural structures for intelligent agents presents a unique challenge when operating in real-time domains. Neurons in our brains have different response properties, firing rates, and propagation lengths, creating noise that cannot be reliably decoded. This research explores the strengths and limitations of LIF spiking neuron ensembles for application in OpenAI virtual environments. Topics discussed include how we represent arbitrary environmental signals from multiple senses, choosing between equally viable actions in a given scenario, and how one can create a generic model that can learn and operate in a verity of situations.

Video games have proved to be a very defying laboratory to study machine-learning techniques, such as Deep Reinforcement Learning (DRL) algorithms. This paper presents a new approach for a DRL-based agent trained through Deep Q-Network (DQN) technique to perform free kicks in FIFA 18 game. The main motivation for choosing this case study is the fact that, like in many situations of the real life, FIFA represents a stochastic environment. Coping with this task, the main contributions of the present paper consist on: inspired on the OpenAI Gym and on the OpenAI Universe platforms, implementing a new user-friendly interface (in terms of portability and use simplicity) to connect the learning module with the 3D FIFA's game environment; implementing a DRL-based agent for free kicks in FIFA that uses two distinct data representations retrieved from lower cost computational procedures. The results were validated through two evaluative parameters: score of well succeed kicks and training time.