Reinforcement Learning
Adaptive Auxiliary Task Weighting for Reinforcement Learning
Xingyu Lin, Harjatin Baweja, George Kantor, David Held
Neural Information Processing Systems http://nips.cc/
Generalized Off-Policy Actor-Critic
Shangtong Zhang, Wendelin Boehmer, Shimon Whiteson
Neural Information Processing Systems http://nips.cc/
A Unifying View of Optimism in Episodic Reinforcement Learning
In this paper, we provide a new framework for studying this class of algorithms. Optimistic algorithms are built upon the principle of "optimism in the face of uncertainty" (OFU).
Appendix B, we provide sufficient conditions for Assumption 1 that were mentioned in the main
In Appendix A we introduce some basic definitions that are needed for our theoretical results. In Appendix C and Appendix D we prove the error bounds for PPI and PQI. All the other dynamics are preserved. Rewards are 0 for the absorbing action and unchanged elsewhere. Algorithm 1 and 2. As some of the notations is actually a function of the MDP, we clarify the usage Recall the definition of semi-norm of any function of state-action pairs.
Provably Good Batch Reinforcement Learning Without Great Exploration
Batch reinforcement learning (RL) is important to apply RL algorithms to many high stakes tasks.
Language as an Abstraction for Hierarchical Deep Reinforcement Learning
YiDing Jiang, Shixiang (Shane) Gu, Kevin P. Murphy, Chelsea Finn
Solving complex, temporally-extended tasks is a long-standing problem in reinforcement learning (RL).
Conservative Q-Learning for Offline Reinforcement Learning A viral Kumar
Effectively leveraging large, previously collected datasets in reinforcement learning (RL) is a key challenge for large-scale real-world applications. Offline RL algorithms promise to learn effective policies from previously-collected, static datasets without further interaction.
Learning Continuous Control Policies by Stochastic Value Gradients
Nicolas Heess, Gregory Wayne, David Silver, Timothy Lillicrap, Tom Erez, Yuval Tassa
We present a unified framework for learning continuous control policies using backpropagation. It supports stochastic control by treating stochasticity in the Bellman equation as a deterministic function of exogenous noise. The product is a spectrum of general policy gradient algorithms that range from model-free methods with value functions to model-based methods without value functions. We use learned models but only require observations from the environment instead of observations from model-predicted trajectories, minimizing the impact of compounded model errors. We apply these algorithms first to a toy stochastic control problem and then to several physics-based control problems in simulation. One of these variants, SVG(1), shows the effectiveness of learning models, value functions, and policies simultaneously in continuous domains.
Supplementary Material: Discovering Reinforcement Learning Algorithms Junhyuk Oh Matteo Hessel Wojciech M. Czarnecki Zhongwen Xu Hado van Hasselt Satinder Singh David Silver DeepMind
In tabular grid worlds, object locations are randomised across lifetimes but fixed within a lifetime. There are two different action spaces. The other version has only 9 movement actions. The episode terminates after a fixed number of steps (i.e., chain length), which is There is no state aliasing because all states are distinct. We trained LPGs by simulating 960 parallel lifetimes (i.e., batch size for meta-gradients), each of Rectified linear unit (ReLU) was used as activation function throughout the experiment.