Towards continual task learning in artificial neural networks: current approaches and insights from neuroscience

The innate capacity of humans and other animals to learn a diverse, and often interfering, range of knowledge and skills throughout their lifespan is a hallmark of natural intelligence, with obvious evolutionary motivations. In parallel, the ability of artificial neural networks (ANNs) to learn across a range of tasks and domains, combining and re-using learned representations where required, is a clear goal of artificial intelligence. This capacity, widely described as continual learning, has become a prolific subfield of research in machine learning. Despite the numerous successes of deep learning in recent years, across domains ranging from image recognition to machine translation, such continual task learning has proved challenging. Neural networks trained on multiple tasks in sequence with stochastic gradient descent often suffer from representational interference, whereby the learned weights for a given task effectively overwrite those of previous tasks in a process termed catastrophic forgetting. This represents a major impediment to the development of more generalised artificial learning systems, capable of accumulating knowledge over time and task space, in a manner analogous to humans. A repository of selected papers and implementations accompanying this review can be found at https://github.com/mccaffary/ Several definitions of intelligence have emerged in machine learning literature in recent years (Legg & Hutter, 2007; Tenenbaum et al., 2011). Although many of these differ in particularities, one consistent requirement is that an intelligent agent must have the capacity to accumulate and maintain task proficiency across experiences (Hinton et al., 1986; Hassabis et al., 2017). In contrast to biological agents, existing neural network approaches have proved lacking in this regard, with sequential task learning often resulting in catastrophic forgetting of previously encountered tasks (French, 1999; McCloskey & Cohen, 1989; Ratcliff, 1990).