LLEDA -- Lifelong Self-Supervised Domain Adaptation

Humans and animals can continuously acquire new information over their lifetime without catastrophically forgetting the prior knowledge learned. This ability to continually learn over time by accommodating new knowledge while retaining the previously learned knowledge is referred to as lifelong or continual learning (in our paper, we will continue to refer to it as lifelong learning). However, artificial neural networks lack these capabilities as new information interferes with previously learned knowledge and sometimes the old knowledge completely gets overwritten by the new one, leading to impaired performance [8]. The root cause of catastrophic forgetting is that learning necessitates changes in the weights of a neural network, however, these changes also result in the forgetting of previous learning. The focus of this paper is on lifelong domain adaptation, in which the model is trained on multiple sequential domains, continuously adapting to new domains with changing distributions as they become available, while maintaining its knowledge of previously encountered domains. Domain adaptation (DA) methods based on deep learning have received significant attention in recent years for mitigating the domain shift from the training domain to the inference domain [9, 10, 11, 12], and have even been suggested as transformative technologies in settings such as agriculture [13, 14] and arts [15].