In particular, using stochastic gradients in MAML update steps is crucial for RL problems since computation of exact gradients requires access to a large number of possible trajectories.

Behavioral cloning has proven to be effective for learning sequential decisionmaking policies fromexpertdemonstrations.

Behaviour cloning (BC) is a classic algorithm to imitate expert demonstrations [7], which uses supervised learning to greedily match the expert behaviour at demonstrated expert states. Due to environmentstochasticity,covariateshift,andpolicyapproximationerror,theagentmaydriftaway from the expert state distribution and ultimately fail to mimic the demonstrator [8].

In this paper, we consider whether learned optimization can help overcome these problems.

Inthesecondexample, vacuuming the floors of a house has certain risks, but the consequences of optimizing the wrong rewardfunction arearguably muchlesssignificant.