While training with static demonstrations has shown some promise, we show that such methods fall short for controlling real GUIs due to their failure to deal with real world stochasticity and non-stationarity not captured in static observational data.

As advocated by [39, 38, 18], procedurally generated environments are a promising direction for testing systematic generalization of RL agents.

The checklist follows the references. Please do not modify the questions and only use the provided macros for your answers. Checklist section does not count towards the page limit. Do the main claims made in the abstract and introduction accurately reflect the paper's Did you describe the limitations of your work? Did you discuss any potential negative societal impacts of your work?

In thispaper,weinvestigatewhether onesuchobjective,theWasserstein-1 distance between a policy's state visitation distribution and a target distribution, can be utilized effectivelyforreinforcement learning (RL)tasks.

Through reinforcement learning (RL), we have made massive strides towards solving tasks that haveasingle goal. However,inthe multi-task domain, where an agent needs to reach multiple goals, the choice of training goals can largely affectsampleefficiency. Whenbiologicalagentslearn,thereisoftenanorganized and meaningful order to which learning happens. Inspired by this, we propose setting up an automatic curriculum for goals that the agent needs to solve.