A Experimental Details

A.1 Environments and T asks We provide the details about the environments and tasks used in our experiments in Table 1. T able 1: Environments and tasks from the DeepMind control suite [29] used in our experiments. Near-expert data: Same as the above near-expert dataset, but we only include 2M steps experience (2K episodes in total) for each task. Goal-MLP Training We adapt the training of Goal-MLP to make it learn to reach goals with varying time budgets. MaskDP is designed to be accessible to the RL research community.