Supplementary Materials: Training Stronger Baselines for Learning to Optimize Tianlong Chen

L2O-DM-CL donates the enhanced L2O-DM with our proposed curriculum learning technique. All learnable optimizers are trained with 5000 epochs. The results are presented in figure A2. We observe that the model trained by curriculum learning outperforms the two baselines (i.e., L2O-DM and L2O-DM-AUG) with Curves are the average of ten runs. Evaluation performance of our enhanced L2O and previous SOT As (i.e., log training loss v.s.