Supplementary Material: Continuous MDP Homomorphisms and Homomorphic Policy Gradient