Cross-lingual Lifelong Learning

The longstanding goal of multi-lingual learning has been to develop a universal cross-lingual model that can withstand the changes in multilingual data distributions. There has been a large amount of work to adapt such multilingual models to unseen target languages. However, the majority of work in this direction focuses on the standard one-hop transfer learning pipeline from source to target languages, Figure 1: An overview of CCL: We use an example whereas in realistic scenarios, new languages of a non-stationary datastream moving from high to can be incorporated at any time in a sequential low resource languages. Each bold and dashed box manner. In this paper, we present a principled represents either a training or test data instance being Cross-lingual Continual Learning (CCL) evaluation fine-tuned or evaluated on, respectively. To support this paradigm, where we analyze different categories problem setup, we evaluate the cross-lingual capabilities of approaches used to continually adapt of continual approaches. Those capabilities include to emerging data from different languages. We knowledge preservation on old languages, accumulation provide insights into what makes multilingual to the current language, and generalization to sequential learning particularly challenging.