Reinforcement Learning
Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL Andrew Wagenmaker
Such direct sim2real transfer is not guaranteed to succeed, however, and in cases where it fails, it is unclear how to best utilize the simulator. In this work, we show that in many regimes, while direct sim2real transfer may fail, we can utilize the simulator to learn a set of exploratory policies which enable efficient exploration in the real world.
Tackling Heavy-Tailed Rewards in Reinforcement Learning with Function Approximation: Minimax Optimal and Instance-Dependent Regret Bounds Jiayi Huang 1 Han Zhong 1 Liwei Wang
Our contributions are summarized as follows.
Distributional Model Equivalence for Risk-Sensitive Reinforcement Learning Tyler Kastner
We consider the problem of learning models for risk-sensitive reinforcement learning.
Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning Shuguang Y u
Before deploying any newly developed policy, it is important to assess its impact. In many high-stakes domains, it is risky or unethical to implement such policies directly for online evaluation.
SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions Zizhao Wang
Unsupervised skill discovery carries the promise that an intelligent agent can learn reusable skills through autonomous, reward-free environment interaction.
An Analytical Study of Utility Functions in Multi-Objective Reinforcement Learning
In this paper, we contribute to both theoretical analyses.
One Demonstration is Enough to Learn Robot Policies
Sequential decision-making problems typically require significant human supervision and data.
Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learning Gen Li CUHK Wenhao Zhan Princeton Jason D. Lee Princeton Y uejie Chi CMU Y uxin Chen
This paper studies tabular reinforcement learning (RL) in the hybrid setting, which assumes access to both an offline dataset and online interactions with the unknown environment. A central question boils down to how to efficiently utilize online data to strengthen and complement the offline dataset and enable effective policy fine-tuning. Leveraging recent advances in reward-agnostic exploration and of-fline RL, we design a three-stage hybrid RL algorithm that beats the best of both worlds -- pure offline RL and pure online RL -- in terms of sample complexities. The proposed algorithm does not require any reward information during data collection. Our theory is developed based on a new notion called single-policy partial concentrability, which captures the trade-off between distribution mismatch and miscoverage and guides the interplay between offline and online data.