Our approach combines three key ideas: (1) value-demand augmentation, (2) action-space approximate dynamic programming, and (3) time-space rounding.

RL benchmark tailored for embodied pixel-based RL.

In contrast, we design a discrete-time algorithm and derive its convergence rate solely under weakly monotone conditions.

Offline reinforcement learning (RL) aims to learn optimal policies from offline datasets, where the parameterization of policies is crucial but often overlooked.