One is More: Diverse Perspectives within a Single Network for Efficient DRL

Deep reinforcement learning has achieved remarkable performance in various domains by leveraging deep neural networks for approximating value functions and policies. However, using neural networks to approximate value functions or policy functions still faces challenges, including low sample efficiency and overfitting. In this paper, we introduce OMNet, a novel learning paradigm utilizing multiple subnetworks within a single network, offering diverse outputs efficiently. We provide a systematic pipeline, including initialization, training, and sampling with OMNet. OMNet can be easily applied to various deep reinforcement learning algorithms with minimal additional overhead. Through comprehensive evaluations conducted on MuJoCo benchmark, our findings highlight OMNet's ability to strike an effective balance between performance and computational cost. Deep reinforcement learning, as evidenced by various studies (Mnih et al., 2015; Silver et al., 2017; Jumper et al., 2021), has demonstrated its versatility across numerous domains, surpassing the scope of traditional reinforcement learning. This achievement can be attributed to the remarkable function approximation prowess exhibited by deep neural networks. Within the realm of deep reinforcement learning, neural networks play a pivotal role in approximating value functions, policy functions, and other critical components.