Reinforcement Learning (RL) has achievedimpressiveprogress inrecent years, such asresults in Atari [24] and Go [28] in which RL agents even perform better than human beings.

Our framework is the infinite-horizon discounted Markov Decision Process (MDP).

Neural Information Processing Systems http://nips.cc/

In this work, we introduce a framework for policy optimization based on mirror descent that naturally accommodates general parameterizations. The policy class induced by our scheme recovers known classes, e.g., softmax, and generates new ones depending on the choice of mirror map.

We explore representing complex actions as natural language instructions.

How to design efficient algorithms is a central problem for reinforcement learning (RL).

In this paper, we take a step toward narrowing the gap between model-based and model-free RL methods.

Diversity in equally-performing policies.We show that different neighborhoods correspond to different post-update return distributions and agent behaviors. We discover that at equal average returns, different policies obtained by the same deep RL algorithm may in fact have substantially different distributional profiles, as measured by statistics of the post-update return distribution.