LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR

When new languages need to be integrated into a multilingual ASR system, a naive Recent years have witnessed significant progress in multilingual approach is to fine-tune the ASR model using data from these automatic speech recognition (ASR), driven by the emergence new languages. Unfortunately, this often results in catastrophic of end-to-end (E2E) models and the scaling of multilingual forgetting, referring to the phenomenon that the recognition performance datasets. Despite that, two main challenges persist in multilingual of base languages tends to decline. To solve the above ASR: language interference and the incorporation of problem, Li et al. [26] proposes lifelong learning [27] solution new languages without degrading the performance of the existing which remedies the language interference problem by mixing ones. This paper proposes LoRA-Whisper, which incorporates base language data and new language data. However, this approach LoRA matrix into Whisper for multilingual ASR, is inefficient and time-consuming. Libera et al. [28] explores effectively mitigating language interference. Furthermore, by various continual learning methods [29-34] to address leveraging LoRA and the similarities between languages, we the issue of catastrophic forgetting. While these approaches can achieve better performance on new languages while upholding have helped alleviate the problem, it still persists.