A Multi-Head Model for Continual Learning via Out-of-Distribution Replay

Many approaches have been proposed to deal with catastrophic forgetting (CF) in CIL. Most methods incrementally construct a single classifier for all classes of all tasks in a single head network. To prevent CF, a popular approach is to memorize a small number of samples from previous tasks and replay them during training of the new task. However, this approach still suffers from serious CF as the parameters learned for previous tasks are updated or adjusted with only the limited number of saved samples in the memory. This paper proposes an entirely different approach that builds a separate classifier (head) for each task (called a multi-head model) using a transformer network, called MORE. Instead of using the saved samples in memory to update the network for previous tasks/classes in the existing approach, MORE leverages the saved samples to build a task specific classifier (adding a new classification head) without updating the network learned for previous tasks/classes. The model for the new task in MORE is trained to learn the classes of the task and also to detect samples that are not from the same data distribution (i.e., out-of-distribution (OOD)) of the task. This enables the classifier for the task to which the test instance belongs to produce a high score for the correct class and the classifiers of other tasks to produce low scores because the test instance is not from the data distributions of these classifiers. Experimental results show that MORE outperforms state-of-the-art baselines and is also naturally capable of performing OOD detection in the continual learning setting. Continual learning (CL) is a learning paradigm in which a system learns a sequence of tasks sequentially and accumulates the knowledge learned in the process (Chen & Liu, 2018). A main challenge of CL is how to adapt the existing knowledge in learning the new task without causing catastrophic forgetting (CF) (McCloskey & Cohen, 1989). CF refers to the phenomenon that the system forgets some of the previous knowledge after learning the new task due to modifications to model parameters learned for previous tasks. This paper proposes a novel method for the challenging CL setting of class incremental learning (CIL) (Rebuffi et al., 2017; van de Ven & Tolias, 2019). At any time, the system is expected to be able to classify a test instance x to one of the classes that have been learned so far without any information about the task it belongs to. The proposed approach is a memory-based method (also called a replay-based method). In this method, the system saves a small fraction of training samples in a memory buffer of a fixed size after learning each task.