A Theoretical Analysis

This section contains the theoretical analysis of the loss functions of offline experience replay (Proposition 2), augmented experience replay (Proposition 3), and online experience replay with reservoir sampling (Proposition 1). At each iteration t, t = 1,..T, a batch of data is sampled from the incoming task, B Note 3: Consider a balanced continual learning dataset (e.g., Split-CIFAR100, Split-Mini-ImageNet) where |D Note 4: Consider general continual learning datasets. Table 3 lists the image size, the number of classes, the number of tasks, and data size per task of the four CL benchmarks. C.1 Continual Learning Implementation The hyperparameter settings are summarized in Table 4. All models are optimized using vanilla SGD.