Implicit Distributional Reinforcement Learning: Appendix A Proof of Lemma

Additional ablation studies on Ant is shown in Figure 1a for a thorough comparison. We show in Figure 1 the visualization of the late stage policy of one seed from Walker2d-v2 environment.