Technology
OntheEffectivenessofLipschitz-Driven RehearsalinContinualLearning
Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small memory buffer; subsequently, they repeatedly optimize on the latter to prevent catastrophic forgetting. This work draws attention to a hidden pitfallofthis widespread practice: repeated optimization onasmall pool of data inevitably leads to tight and unstable decision boundaries, which are a major hindrance to generalization.
Distributional Gradient Matching for Learning Uncertain Neural Dynamics Models
Differential equations in general and neural ODEs in particular are an essential technique in continuous-time system identification. While many deterministic learning algorithms have been designed based on numerical integration via the adjoint method, many downstream tasks such as active learning, exploration in reinforcement learning, robust control, or filtering require accurate estimates of predictive uncertainties.