Learning to Learn without Forgetting using Attention

Continual learning (CL) refers to the ability to continually learn over time by accommodating new knowledge while retaining previously learned experience. While this concept is inherent in human learning, current machine learning methods are highly prone to overwrite previously learned patterns and thus forget past experience. Instead, model parameters should be updated selectively and carefully, avoiding unnecessary forgetting while optimally leveraging previously learned patterns to accelerate future learning. Since hand-crafting effective update mechanisms is difficult, we propose meta-learning a transformer-based optimizer to enhance CL. This meta-learned optimizer uses attention to learn the complex relationships between model parameters across a stream of tasks, and is designed to generate effective weight updates for the current task while preventing catastrophic forgetting on previously encountered tasks. Evaluations on benchmark datasets like SplitMNIST, RotatedMNIST, and SplitCIFAR-100 affirm the efficacy of the proposed approach in terms of both forward and backward transfer, even on small sets of labeled data, highlighting the advantages of integrating a meta-learned optimizer within the continual learning framework. Continual learning (CL), or lifelong learning, refers to the ability of a machine learning system to continuously acquire knowledge from a stream of learning tasks while retaining previously learned experience (Cossu et al., 2021). CL involves a non-stationary stream of tasks where the data distribution evolves over time (Son et al., 2023). For instance, consider an autonomous vehicle driving across different countries under varying weather conditions and road surfaces. The vehicle must constantly adapt to changes in the environment by leveraging past knowledge to accelerate the learning process without forgetting how to handle past environments. Conventional neural networks will update all model parameters (weights) when adapting to new tasks, overwriting previously stored information, and therefore tend to gradually forget previously learned knowledge, a phenomenon known as catastrophic forgetting.