Our key insight is that the user's examination and click behavior

Sample efficiencyof ESRL is independent of the chosen risk aversion threshold and quality of the behavior policy.

Policy optimization, i.e. algorithms that learn to make sequential decisions by local search on the agent's policy directly, is a widely used class of algorithms in reinforcement learning [40, 44, 45].

Let 1, , M be K Kis PSD, i 0foralli.

In this paper, we establish an upper bound on the return gap between the oracle expert policy and an optimal decision tree policy. This enables us to recast the DT extraction problem into a novel non-euclidean clustering problem over the local observation and action values space of each agent, with action values as cluster labels and the upper bound on the return gap as clustering loss.

We also study the more general case of an additional entropy regularizer.