A Experimental Details

The episode length is 150 for RoboBin and RoboKitchen and 1000 for RoboY oga. We show all goals in Figure A.1. For both W alker and Quadruped, the success criterion is based on the largest violation across all joints. Global position is not taken into account for the success computation. For reaching task, the success is based on placing the arm in the correct position within 10 cm.