On Training Recurrent Neural Networks for Lifelong Learning

Lifelong Machine Learning considers systems that can learn many tasks (from one or more domains) over a lifetime (Thrun, 1998; Silver et al., 2013). This has several names and manifestations in the literature: incremental learning (Solomonoff, 1989), continual learning (Ring, 1997), explanation-based learning (Thrun, 1996, 2012), never ending learning (Carlson et al., 2010), etc. The underlying idea motivating these efforts is the following: Lifelong learning systems would be more effective at learning and retaining knowledge across different tasks. By using the prior knowledge and exploiting similarity acrosstasks, they would be able to obtain better priors for the task at hand. Lifelong learning techniques are very important for training intelligent autonomous agents that would need to operate and make decisions over extended periods of time. These characteristics arespecially important in the industrial setups where the deployed machine learning models are being updated frequently with new incoming data whose distribution neednot match the data on which the model was originally trained. Lifelong learning is an extremely challenging task for the machine learning models because of two primary reasons: 1. Catastrophic Forgetting: As the model is trained on a new task (or a new data/task distribution), it is likely to forget the knowledge it acquired from the previous tasks (or data distributions). This phenomenon is also known as the catastrophic interference (McCloskey and Cohen, 1989).