The V alue-Equivalence Principle for Model-Based Reinforcement Learning Supplementary Material