r-max
Explicit Planning for Efficient Exploration in Reinforcement Learning
Liangpeng Zhang, Ke Tang, Xin Yao
Neural Information Processing Systems http://nips.cc/
Explicit Planning for Efficient Exploration in Reinforcement Learning
Liangpeng Zhang, Ke Tang, Xin Yao
A straightforward example is as follows.
We will improve the broader impact section by emphasizing the implications of our theoretical
We sincerely thank all the reviewers, and feel really honored to receive such positive and constructive comments. We will mention total variation distance in the appendix, and correct the typo on "Corollary Note that the smooth planning oracle is not needed throughout the paper, and is thus not the "primary It is only used in Sec. We have discussed R-MAX in lines 82-83. By saying "especially model-free ones..." this sentence, we simply meant The works on Q-learning in games you mentioned exactly conquered this issue, with non-trivial efforts. We will address all the grammatical comments/typos in the final version.
Increasingly Cautious Optimism for Practical PAC-MDP Exploration
Zhang, Liangpeng (University of Science and Technology of China) | Tang, Ke (University of Science and Technology of China) | Yao, Xin (University of Birmingham)
Exploration strategy is an essential part of learning agents in model-based Reinforcement Learning. R-MAX and V-MAX are PAC-MDP strategies proved to have polynomial sample complexity; yet, their exploration behavior tend to be overly cautious in practice. We propose the principle of Increasingly Cautious Optimism (ICO) to automatically cut off unnecessarily cautious exploration, and apply ICO to R-MAX and V-MAX, yielding two new strategies, namely Increasingly Cautious R-MAX (ICR) and Increasingly Cautious V-MAX (ICV). We prove that both ICR and ICV are PACMDP, and show that their improvement is guaranteed by a tighter sample complexity upper bound. Then, we demonstrate their significantly improved performance through empirical results.
Learning to Coordinate Efficiently: A Model-based Approach
Brafman, R. I., Tennenholtz, M.
In common-interest stochastic games all players receive an identical payoff. Players participating in such games must learn to coordinate with each other in order to receive the highest-possible value. A number of reinforcement learning algorithms have been proposed for this problem, and some have been shown to converge to good solutions in the limit. In this paper we show that using very simple model-based algorithms, much better (i.e., polynomial) convergence rates can be attained. Moreover, our model-based algorithms are guaranteed to converge to the optimal value, unlike many of the existing algorithms.