Curvature Explains Loss of Plasticity

Loss of plasticity is a phenomenon in which neural networks lose their ability to learn from new experience. Despite being empirically observed in several problem settings, little is understood about the mechanisms that lead to loss of plasticity. In this paper, we offer a consistent explanation for plasticity loss, based on an assertion that neural networks lose directions of curvature during training and that plasticity loss can be attributed to this reduction in curvature. To support such a claim, we provide a systematic empirical investigation of plasticity loss across several continual supervised learning problems. Our findings illustrate that curvature loss coincides with and sometimes precedes plasticity loss, while also showing that previous explanations are insufficient to explain loss of plasticity in all settings. Lastly, we show that regularizers which mitigate loss of plasticity also preserve curvature, motivating a simple distributional regularizer that proves to be effective across the problem settings considered. A longstanding goal of machine learning research is to develop algorithms that can learn continually and cope with unforeseen changes in their environment (Sutton et al., 2007). Current learning algorithms, however, struggle to learn from dynamically changing targets and are unable to adapt gracefully to unforeseen environment changes during the learning process (Abbas et al., 2023, Dohare et al., 2023a, Lyle et al., 2023, Zilly et al., 2021). Such limitations can be seen to be a byproduct of following a supervised learning paradigm that assumes the problem is stationary. Recently, there has been growing recognition of the fact that there are limitations to what can be learned from a fixed and unchanging dataset (Hoffmann et al., 2022) and that there are implicit non-stationarities in many learning problems of interest (Igl et al., 2021). The concept of plasticity has been receiving growing attention in the continual learning literature, where the loss of plasticity--a reduction in the ability to learn new things (Dohare et al., 2023a, Lyle et al., 2023)--has been noted as a critical shortcoming in current learning algorithms. That is, learning algorithms that are performant in the non-continual learning setting often struggle when applied to continual learning problems, exhibiting a striking loss of plasticity such that learning slows down or even halts after successive changes in the learning environment. Such a loss of plasticity can be readily observed in settings where a neural network must continue to learn after changes occur in the observations or targets.