BC formulates imitation learning as a supervised learning problem. It needs no in-environment samples, but it suffers from the covariate shift issue [37], often leading totesttimeperformance degradation.

However, there are still quite a few challenges between the traditional RL research and real-worldtasks.

We characterise the phenomenon empirically, verifying that it is not limited to specific algorithm or environment properties.