AMoreBackgrounds A.1 DistributionalRL Distributional RL [2, 3, 8] is an area of RL that considers the distribution of the cumulative return Zπ(s,a) = P t=0γ