Offline reinforcement learning (RL) aims to learn policies from pre-existing datasets without further interactions, making it a challenging task. Q-learning algorithms struggle withextrapolation errors inofflinesettings, while supervised learning methods are constrained by model expressiveness.

We propose to learn to distinguish reversible from irreversible actions for better informed decision-making in Reinforcement Learning (RL).

A finite episodic Markov decision process (MDP) is a tuple (S,A,H,α,P,r) where S and A are the finite sets of states and actions withS = |S|,A = |A|, H is the (fixed) episode length andα is the initial state distribution.

This is particularly challenging for high-dimensional control tasks, in which there may be a large number of factors that influence the agent's objective.