Both algorithms yield state-of-the-art performance guarantees.

The latter case can be further reduced to adversarial MDP when preferences only depend on the final state.

The proximal policy optimization (PPO) algorithm stands as one of the most prosperous methods in the field of reinforcement learning (RL). Despite its success, the theoretical understanding of PPO remains deficient. Specifically, it is unclear whether PPO or its optimistic variants can effectively solve linear Markov decision processes (MDPs), which are arguably the simplest models in RL with function approximation.

HRL lead to improved sample complexity, and how much of an improvement can it provide? Theoretical work on sample-complexity bound in Machine Learning has been integral to the development of the field.