A Derivations

To achieve learning in deeper networks we have used a curriculum on random and MNIST data. Next, we use a deep network and provide intermediate errors by a ground truth network. Finally, we remove intermediate errors and use the RNN's intermediate predictions that are now close to the ground truth. Figure 12 provides the entire meta test training trajectories for a subset of all configurations. Furthermore, in Figure 13 we show the cumulative accuracy on the first 100 examples.