In contrast, we design a discrete-time algorithm and derive its convergence rate solely under weakly monotone conditions.

Offline reinforcement learning (RL) aims to learn optimal policies from offline datasets, where the parameterization of policies is crucial but often overlooked.

Typically, deep RL algorithms learn a policy in an online trial-and-error fashion using millions to billions of data.