Appendix A Convergence of with the hybrid loss

Before presenting the formal version of Theorem 4.1 and its proof, we introduce some preliminaries. As stated in Theorem 4.1, we assume that both the discriminator class Now we are ready to present a formal version of Theorem 4.1 as follows. By the triangle inequality and Eq.A.12, we obtain λ null By Eq.A.2, Eq.A.11, and Eq.A.14, we have d In this section, we prove Proposition 3.1. In this section, we will give a brief proof of Theorem 4.2, and show that the learning policy can find Suppose the stationary point of the Bellman equation w.r.t the production sample space In this section, we will give a brief proof of Theorem 4.3, and show the convergence of the learning First, we show the monotonic improvement of Q function of the iterated policy by CPED. The Gym-MuJoCo is a commonly used benchmark for offline RL task.