We propose to learn to distinguish reversible from irreversible actions for better informed decision-making in Reinforcement Learning (RL).

A finite episodic Markov decision process (MDP) is a tuple (S,A,H,α,P,r) where S and A are the finite sets of states and actions withS = |S|,A = |A|, H is the (fixed) episode length andα is the initial state distribution.

This is particularly challenging for high-dimensional control tasks, in which there may be a large number of factors that influence the agent's objective.

Thisisbecause, in the traditional analysis, the error bound scales up with this ratio. We show that using pessimistic value estimatesin the low-data regions in Bellman optimality and evaluation back-up can yield more adaptive and stronger guarantees when the concentrability assumption does not hold.

A fundamental difference between our learning problem from standard RL problems is that the realized reward feedback from conversion incrementality ismixed and delayed.