A Algorithm table

We provide an algorithm table that represents HIGL in Algorithm 1. Algorithm 1 Hierarchical reinforcement learning guided by landmarks (HIGL)Input: Goal transition function h, state-goal mapping function ϕ, high-level action frequency m, RND networks θ, θ Initialize empty adjacency matrix M Initialize priority queue Q for n = 1,...,N do Reset the environment and sample the initial state s Sample episode end signal done . Build a graph with the sampled landmarks, a state and a goal. A simulated ball (point mass) starts at the bottom left corner in a " "-shaped maze and aims to reach the top left corner. Its actions correspond to torques applied to joints. This environment has a " "-shaped maze whose size is We define a "success" as being within an L2 distance Each episode is terminated if the agent reaches the goal or after 500 steps.