Lightweight Relevance Grader in RAG

--Retrieval-Augmented Generation (RAG) addresses limitations of large language models (LLMs) by leveraging a vector database to provide more accurate and up-to-date information. When a user submits a query, RAG executes a vector search to find relevant documents, which are then used to generate a response. However, ensuring the relevance of retrieved documents with a query would be a big challenge. T o address this, a secondary model, known as a relevant grader, can be served to verify its relevance. T o reduce computational requirements of a relevant grader, a lightweight small language model is preferred. Its precision is comparable to that of llama-3.1-70b. Large Language Models (LLMs) have achieved exceptional capabilities in various Natural Language Processing (NLP) tasks [1]-[3], demonstrating their ability to absorb and retain vast amounts of knowledge. When responding to specific queries, LLMs often provide informative answers, leveraging the extensive range of information they acquired during their training.