Offline_Distributional_RL__NeurIPS_2021_Submission_ (6)

We give a proof in Appendix A.5. As we discuss in Appendix A.6, we can use this result to obtain First, by Lemma 3.4, we have F Then, by Lemma A.1, with probability at least 1, we have F Note that to show the claim, it suffices to show that for sufficient large, we have ( / 2) c ( s) ( s)+ ( 8 s) . The claim follows by taking the limit k!1 . We first prove a bound on the concentration of the empirical CDF to the true CDF. We proceed by bounding the two terms in the summation.