Reviews: Adaptive Gradient-Based Meta-Learning Methods

The paper presents a new way to analyze multi-task learning algorithms in the online setting by analysing average regret-upper-bound. The authors introduce a two-level algorithm that runs a mirror descent algorithm for each task and adjusts the parameters of the mirror descent before each new task by running another online learning algorithm on the level of tasks. This task-level algorithm is designed to minimize theoretical regret bounds for each task. The authors prove a general bound on the performance of the presented algorithm and show its corollaries for settings with static and dynamic comparators. A version of the algorithm for different task similarity measures is also presented, as well as, an online-to-batch conversion result for learning-to-learn setting with i.i.d.