HyperLoader: Integrating Hypernetwork-Based LoRA and Adapter Layers into Multi-Task Transformers for Sequence Labelling

We use the encoder-decoder T5 model only a small number of parameters is updated to (Raffel et al., 2020) for all experiments to take a downstream task (Houlsby et al., 2019; Stickland advantage of modelling the tasks as sequence-tosequence and Murray, 2019; Karimi Mahabadi et al., tasks. We test our model in seven datasets 2021a). These methods aim to achieve comparable from two Sequence Labelling tasks. The first task performance to full fine-tuning by updating as few is Named Entity Recognition, a valuable tool in parameters as possible. However, a less studied research various real-world scenarios in the era of large language direction related to these methods is whether models such as healthcare and medical research one can perform better than full fine-tuning with (Raza et al., 2022; Hu et al., 2024), Finance fewer parameters (Mao et al., 2022).