A.1 Setup Environments We consider four robotic locomotion and four manipulation environments, all with continuous action spaces. The robotic locomotion environments, based on MuJoCo [27] and OpenAI Gym [3], fall into two categories. Varying reward functions: HalfCheetahRandVel, Walker2DRandVel The HalfCheetahRandVel environment was introduced in Finn et al. [9]. The distribution of tasks is a distribution of HalfCheetah robots with different goal velocities, and remains the same for meta-training and meta-testing. The Walker2DRandVel environment, defined similarly to HalfCheetahRandVel, is found in the codebase for Rothfuss et al. [21].

A.1 Setup Environments We consider four robotic locomotion and four manipulation environments, all with continuous action spaces. The robotic locomotion environments, based on MuJoCo [27] and OpenAI Gym [3], fall into two categories. Varying reward functions: HalfCheetahRandVel, Walker2DRandVel The HalfCheetahRandVel environment was introduced in Finn et al. [9]. The distribution of tasks is a distribution of HalfCheetah robots with different goal velocities, and remains the same for meta-training and meta-testing. The Walker2DRandVel environment, defined similarly to HalfCheetahRandVel, is found in the codebase for Rothfuss et al. [21].

By searching for shared inductive biases across tasks, meta-learning promises to accelerate learning on novel tasks, but with the cost of solving a complex bilevel optimization problem. We introduce and rigorously define the trade-off between accurate modeling and optimization ease in meta-learning. At one end, classic meta-learning algorithms account for the structure of meta-learning but solve a complex optimization problem, while at the other end domain randomized search (otherwise known as joint training) ignores the structure of meta-learning and solves a single level optimization problem. Taking MAML as the representative meta-learning algorithm, we theoretically characterize the trade-off for general nonconvex risk functions as well as linear regression, for which we are able to provide explicit bounds on the errors associated with modeling and optimization. We also empirically study this trade-off for meta-reinforcement learning benchmarks.