W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering

In knowledge-intensive tasks such as open-domain question answering To overcome the limitations of LLMs' parametric knowledge, retrieval (OpenQA), Large Language Models (LLMs) often struggle augmented generation (RAG) [11, 27] is explored, equipping to generate factual answers relying solely on their internal (parametric) LLMs with a retriever to gather necessary evidence from external knowledge. To address this limitation, Retrieval-Augmented sources. Among the two components of RAG, improving the retriever Generation (RAG) systems enhance LLMs by retrieving relevant information is more feasible due to the recent trend of black-box APIs from external sources, thereby positioning the retriever [33] and the high cost and time requirements of fine-tuning opensource as a pivotal component. Although dense retrieval demonstrates LLMs [10]. The retriever, a critical part of RAG, is typically state-of-the-art performance, its training poses challenges due to either a traditional unsupervised retriever like BM25 [38] or a more the scarcity of ground-truth evidence, largely attributed to the high advanced neural retriever, such as dense retrieval [20, 21, 32, 51], costs of human annotation. In this paper, we propose W-RAG by which encodes questions and passages into the same embedding utilizing the ranking capabilities of LLMs to create weakly labeled space and then measures the question-passage relevance score by data for training dense retrievers.