Supplementary Material for " Hierarchical Adaptive Value Estimation for Multi-modal Visual Reinforcement Learning " Yangru Huang

The contents of this supplementary material are organized as follows: Section A provides additional experimental results, including more results with three modalities, performance under dynamic weathers, performance under several challenging or extreme environmental conditions (e.g., increased number of vehicles and dazzling sunlight), results on DeepMind Control Suit, and ablation study of auxiliary losses and the design of re-fusion. Section B provides further discussions related to our approach. This includes a comparison between value-level dynamic fusion and feature-level dynamic fusion supported by empirical results, the advantages of hierarchical bi-level fusion over uni-level fusion, and the relationship and differences between our approach and the value decomposition techniques in multi-agent RL. Section C describes the details of the experimental setup, including network architectures, hyperparameters, and hardware details. Section D states the potential negative societal impacts of our work.