Disentangled Continual Learning: Separating Memory Edits from Model Updates

To mitigate this is hindered by catastrophic forgetting, the tendency issue, continual learning methods employ strategies such as of neural networks to overwrite existing knowledge when (i) regularization, which aims to preserve existing knowledge learning a new task. Existing continual learning methods by limiting the plasticity of selected network weights alleviate this problem through regularisation, parameter [15, 17, 26, 36], (ii) parameter isolation or dynamic architectures, isolation, or rehearsal, and are typically evaluated on benchmarks which effectively solve each task with a dedicated consisting of a handful of tasks. We propose a novel model [6, 33], or (iii) replay, which augments the training conceptual approach to continual classification that aims data with stored samples from past tasks [4, 12, 30, 32]. to disentangle class-specific information that needs to be Most continual learning methods are evaluated on image memorised from the class-agnostic knowledge that encapsulates classification benchmarks in which a discriminative model generalization. We store the former in a buffer that is transferred across tasks that typically involve disjoint sets can be easily pruned or updated when new categories arrive, of classes. We argue that this purely discriminative learning while the latter is represented with a neural network that framework is not conducive to positive forward or backward generalizes across tasks. We show that the class-agnostic transfer. Supervised classification networks tend to preserve network does not suffer from catastrophic forgetting and by only the features that are relevant for predicting the output leveraging it to perform classification, we improve accuracy labels in the training data [11, 35].