Instead of using an explicit reward function, Inverse Reinforcement Learning (IRL; Ng et al., 2000) seeks to recover the reward by observing anexpert,e.g.,anhuman whoalready knowshowtoperform atask. However,most existing IRL algorithms assume that the transition model, and in some cases, the expert's policy, areknown.

We study the problem of reward shaping to accelerate the training process of a reinforcement learning agent.

The goal of reinforcement learning (RL) [39] is to maximize the expected total reward by taking actions according toapolicyinastochastic environment, whichismodelled asaMarkovdecision process (MDP) [4]. To obtain an optimal policy, one popular method is the direct maximization of the expected total reward via gradient ascent, which is referred to as the policy gradient (PG) method [40,47].

Semantic cues and statistical regularities in real-world environment layouts can improve efficiency for navigation in novel environments.

Inaddition, wehypothesize thatthestrong performance of the one-step algorithm is due to a combination of favorable structure in the environmentandbehaviorpolicy.