This project is maintained by Jiwei Li. This repo will continue to be updated. After training, the trained models will be saved in save_t_given_s/model*. Decoding given a pre-trained generative model. The pre-trained model doesn't have to be a vanila Seq2Seq model (for example, it can be a trained model from adversarial learning).

Scalable and effective exploration remains a key challenge in reinforcement learning (RL). While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high-dimensional deep RL scenarios. As such, most contemporary RL relies on simple heuristics such as epsilon-greedy exploration or adding Gaussian noise to the controls. This paper introduces Variational Information Maximizing Exploration (VIME), an exploration strategy based on maximization of information gain about the agent's belief of environment dynamics. We propose a practical implementation, using variational inference in Bayesian neural networks which efficiently handles continuous state and action spaces. VIME modifies the MDP reward function, and can be applied with several different underlying RL algorithms. We demonstrate that VIME achieves significantly better performance compared to heuristic exploration methods across a variety of continuous control tasks and algorithms, including tasks with very sparse rewards.

Learning to reinforcement learn by Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, Matt Botvinick In recent years deep reinforcement learning (RL) systems have attained superhuman performance in a number of challenging task domains. However, a major limitation of such applications is their demand for massive amounts of training data. A critical present objective is thus to develop deep RL methods that can adapt rapidly to new tasks. In the present work we introduce a novel approach to this challenge, which we refer to as deep meta-reinforcement learning. Previous work has shown that recurrent networks can support meta-learning in a fully supervised context.

In recent years deep reinforcement learning (RL) systems have attained superhuman performance in a number of challenging task domains. However, a major limitation of such applications is their demand for massive amounts of training data. A critical present objective is thus to develop deep RL methods that can adapt rapidly to new tasks. In the present work we introduce a novel approach to this challenge, which we refer to as deep meta-reinforcement learning. Previous work has shown that recurrent networks can support meta-learning in a fully supervised context. We extend this approach to the RL setting. What emerges is a system that is trained using one RL algorithm, but whose recurrent dynamics implement a second, quite separate RL procedure. This second, learned RL algorithm can differ from the original one in arbitrary ways. Importantly, because it is learned, it is configured to exploit structure in the training domain. We unpack these points in a series of seven proof-of-concept experiments, each of which examines a key aspect of deep meta-RL. We consider prospects for extending and scaling up the approach, and also point out some potentially important implications for neuroscience.

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

A pioneer in machine learning from the University of Alberta is teaming up with the Royal Bank of Canada on artificial intelligence research. Richard Sutton, a professor at the school's department of computer science and a graduate of the University of Massachusetts, will advise the bank's machine learning research division and collaborate with RBC's second AI research lab, to be located in Edmonton. Sutton specializes in the same branch of machine learning that Google's AlphaGo computer program used, in part, to beat one of the highest-ranking professional players of the board game Go -- until recently, a notoriously difficult game for computers to play. The announcement is the latest in a string of AI-related partnerships, acquisitions and investments that have been struck in Canada in recent months -- the most high-profile of which have involved Facebook and Google, which have been in a fierce competition for access to talent. For over three decades, Sutton has specialized in reinforcement learning. In this branch of machine learning, an algorithm is designed to receive either a reward or penalty based on its behaviour, and learns to make choices that will result in the most reward -- and, hopefully, most desired behaviour -- over time.

The panel so titled shows the performance of the reinforcement learning algorithm doing dynamic channel allocation in cellular telephones. To learn more about reinforcement learning, please read Reinforcement Learning for Dynamic Channel Allocation in Cellular Telephone Systems.

How to Make Software Agents Do the Right Thing: An Introduction to Reinforcement Learning [html][postscript] (to appear in Dr. Dobbs)

Abstract: Deep learning algorithms have recently appeared that pretrain hidden layers of neural networks in unsupervised ways, leading to state-of-the-art performance on large classification problems. These methods can also pretrain networks used for reinforcement learning. However, this ignores the additional information that exists in a reinforcement learning paradigm via the ongoing sequence of state, action, new state tuples. This paper demonstrates that learning a predictive model of state dynamics can result in a pretrained hidden layer structure that reduces the time needed to solve reinforcement learning problems. After training for 0 minutes: Your browser does not support the video tag.