Studying Generalization on Memory-Based Methods in Continual Learning

One of the objectives of Continual Learning is to learn new concepts continually over a stream Despite successful results, previous works have argued that of experiences and at the same time avoid catastrophic memory-based methods are prone to overfitting (Lopez-Paz forgetting. To mitigate complete knowledge & Ranzato, 2017; Verwimp et al., 2021). By only storing a overwriting, memory-based methods store subset of previous distributions, the model only reinforces a percentage of previous data distributions to be concepts and ideas that are present in the buffer, depending used during training. Although these methods on how much previous distributions are represented. To produce good results, few studies have tested reinforce useful concepts, the buffer should accurately represent their out-of-distribution generalization properties, the whole training distribution. However, if the buffer as well as whether these methods overfit the replay represents only a small percentage of the training distribution, memory. In this work, we show that although it will start learning spurious correlations and will lose these methods can help in traditional indistribution its generalization capabilities.