Supplementary Material for " Hierarchical Adaptive Value Estimation for Multi-modal Visual Reinforcement Learning " Y angru Huang