environmental step
A Appendix
In appendix, we provide some additional results in Section A.1, more implementation details in To compare the stability of training, we didn't early-stop the training process even if the loss of some tasks already exploded. MTRL training compared with both variants, demonstrating the effectiveness of the PaCo design. MT50 is a more complex benchmark in Meta-World containing 50 different manipulation tasks (including the MT10 tasks). Therefore it's hard to determine if the policy has reached to the optimal. The results are shown in Figure 8.
Efficient Multi-Task and Transfer Reinforcement Learning with Parameter-Compositional Framework
Sun, Lingfeng, Zhang, Haichao, Xu, Wei, Tomizuka, Masayoshi
In this work, we investigate the potential of improving multi-task training and also leveraging it for transferring in the reinforcement learning setting. We identify several challenges towards this goal and propose a transferring approach with a parameter-compositional formulation. We investigate ways to improve the training of multi-task reinforcement learning which serves as the foundation for transferring. Then we conduct a number of transferring experiments on various manipulation tasks. Experimental results demonstrate that the proposed approach can have improved performance in the multi-task training stage, and further show effective transferring in terms of both sample efficiency and performance.
Optimal Use of Experience in First Person Shooter Environments
Although reinforcement learning has made great strides recently, a continuing limitation is that it requires an extremely high number of interactions with the environment. In this paper, we explore the effectiveness of reusing experience from the experience replay buffer in the Deep Q-Learning algorithm. We test the effectiveness of applying learning update steps multiple times per environmental step in the VizDoom environment and show first, this requires a change in the learning rate, and second that it does not improve the performance of the agent. Furthermore, we show that updating less frequently is effective up to a ratio of 4:1, after which performance degrades significantly. These results quantitatively confirm the widespread practice of performing learning updates every 4th environmental step.