Reviews: Online-Within-Online Meta-Learning

This work proposes algorithms for the online-within-online meta-learning setting as oppposed to the more prevalent statistical setting. In this particular meta-learning setting tasks arrive sequentially manner (outer loop) and then the learning per task itself happens in an online fashion. The aim is to have low average regret over tasks. The inner loop optimization is done via Online Mirror Descent (OMD). The inner algorithm design is carefully chosen to provide good approximations of (sub)-gradients of the outer meta objective.