Towards a RAG-based Summarization Agent for the Electron-Ion Collider

The Electron Ion Collider (EIC): The EIC is the next generation large-scale scientific project that is anticipated to become operational within the next decade [1]. The EIC User group is a fast-evolving community consisting of more than 1,400 physicists from over 38 countries around the world. Book keeping has been a significant challenge in such large collaborative experiments; specifically, in the case of large-scale physics experiments, one faces significant challenges in coordinating information curation between numerous working groups within the collaboration. New collaborators in such a large-scale experiment often feel overwhelmed when they start. For example, when conducting experiments and collecting data, the institutions involved in the collaboration need to comprehend and review a vast amount of documentation during their respective shifts, which can be daunting for the beginner. In this context, having a virtual assistant available to support shift takers would be immensely beneficial. Fine tuning of Large Language Models (LLMs): Fine-tuning an LLM involves refining its abilities and performance in specific tasks or domains by training it further in domain-specific datasets after pretraining, improving effectiveness without retraining the entire model [10, 12]. However, conventional fine-tuning requires substantial computational power and time, posing challenges, especially for extensive models such as GPT-3 (175-B parameters)[3] and Meta LLaMA2 (13-B parameters) [13]. Despite the advent of Low-Rank Adaptation (LoRA) which enables efficient fine-tuning, even on consumer-grade GPUs, fine-tuning remains computationally intensive.