Provably Efficient Offline Reinforcement Learning in Regular Decision Processes

Most reinforcement learning (RL) algorithms hinge on the Markovian assumption, i.e. that the underlying system transitions and rewards are Markovian in some natural notion of (observable)