Supplementary Material for " Hierarchical Adaptive Value Estimation for Multi-modal Visual Reinforcement Learning " Y angru Huang

The contents of this supplementary material are organized as follows: Section A provides additional experimental results, including more results with three modalities, performance under dynamic weathers, performance under several challenging or extreme environmental conditions ( e.g., increased number of vehicles and dazzling sunlight), results on DeepMind Control Suit, and ablation study of auxiliary losses and the design of re-fusion. Section B provides further discussions related to our approach. This includes a comparison between value-level dynamic fusion and feature-level dynamic fusion supported by empirical results, the advantages of hierarchical bi-level fusion over uni-level fusion, and the relationship and differences between our approach and the value decomposition techniques in multi-agent RL. Section C describes the details of the experimental setup, including network architectures, hyper-parameters, and hardware details. Section D states the potential negative societal impacts of our work.