Continual learning: a feature extraction formalization, an efficient algorithm, and barriers

While this setup has enjoyed a lot of attention in the applied community, there hasn't be theoretical work that even formalizes the desired guarantees.