Continual Learning with Synthetic Boundary Experience Blending

Abstract--Continual learning (CL) seeks to mitigate catastrophic forgetting when models are trained with sequential tasks. A common approach, experience replay (ER), stores past exemplars but only sparsely approximates the data distribution, yielding fragile and oversimplified decision boundaries. We address this limitation by introducing synthetic boundary data (SBD), generated via differential-privacy-inspired noise into latent features to create boundary-adjacent representations that implicitly regularize decision boundaries. Building on this idea, we propose Experience Blending (EB), a framework that jointly trains on exemplars and SBD through a dual-model aggregation strategy. EB has two components: (1) latent-space noise injection to synthesize boundary data, and (2) end-to-end training that jointly leverages exemplars and SBD. Extensive experiments on CIF AR-10, CIF AR-100, and Tiny ImageNet demonstrate consistent accuracy improvements of 10%, 6%, and 13%, respectively, over strong baselines. I. Introduction Deep neural networks (DNNs) achieve remarkable performance across domains but typically rely on large, static datasets. In practice, however, data arrive sequentially, and models must adapt without retraining from scratch. Transfer learning [1] offers a cost-effective way to fine-tune models on new data, but repeated fine-tuning leads to catastrophic forgetting--the rapid degradation of previously learned knowledge as new tasks are introduced. Continual Learning (CL) [2], [3] addresses this challenge by enabling models to learn from a stream of tasks while retaining prior knowledge, thereby reducing performance loss on earlier tasks.