Appendices

Algorithm 1 Curriculum Offline Imitation Learning (COIL) Require: Offline dataset D, number of trajectories picked at each curriculum N, moving window of the return filter α, number of training iteration L, batch size B, number of pre-train times T, and the learning rate η. Initialize the return filter V = 0. if D is collected by a single policy then Do pre-training for T times using BC. B.1 Proof for Theorem 1 We introduce useful lemmas before providing our proof. Therefore, we have the following proposition. Let Π be the set of all deterministic policy and |Π|= |A||S|.