A Additional Experimental Results

Reward curves for TOP-RAD and RAD on pixel-based tasks from the DM Control Suite are shown in Figure 7. Figure 7: Results across 10 seeds for DM Control tasks. Each individual run was performed on a single GPU and lasted between 3 and 18 hours, depending on the task and GPU model. The procedures for updating the critics and the actor for TOP-TD3 are described in detail in Algorithm 2 and Algorithm 3. Algorithm 2: UpdateCritics In order to enable adaptation, we make use of an approach inspired by recent results in the model selection for contextual bandits literature. Bandit problems, the "arm" choices in the model selection setting are not stationary arms, but learning algorithms. The objective is to choose in an online manner, the best algorithm for the task at hand.The In figure 5, Ant-v2 we show this to be the case.