Although LLMs have substantial potential in financial applications, there are notable challenges in using pre-trained models to Extraction and interpretation of intricate information from unstructured extract information from financial documents outside their training text data arising in financial applications, such as earnings data while also reducing hallucination [7, 8]. Financial documents call transcripts, present substantial challenges to large language typically contain domain-specific language, multiple data formats, models (LLMs) even using the current best practices to use Retrieval and unique contextual relationships that general purpose-trained Augmented Generation (RAG) (referred to as VectorRAG LLMs do not handle well. In addition, extracting consistent and techniques which utilize vector databases for information retrieval) coherent information from multiple financial documents can be due to challenges such as domain specific terminology and complex challenging due to variations in terminology, format, and context formats of the documents. We introduce a novel approach based across different textual sources. The specialized terminology and on a combination, called HybridRAG, of the Knowledge Graphs complex data formats in financial documents make it difficult for (KGs) based RAG techniques (called GraphRAG) and VectorRAG models to extract meaningful insights, in turn, causing inaccurate techniques to enhance question-answer (Q&A) systems for information predictions, overlooked insights, and unreliable analysis, which extraction from financial documents that is shown to be ultimately hinder the ability to make well-informed decisions.

Retrieval augmented models show promise in enhancing traditional language models by improving their contextual understanding, integrating private data, and reducing hallucination. However, the processing time required for retrieval augmented large language models poses a challenge when applying them to tasks that require real-time responses, such as composition assistance. To overcome this limitation, we propose the Hybrid Retrieval-Augmented Generation (HybridRAG) framework that leverages a hybrid setting that combines both client and cloud models. HybridRAG incorporates retrieval-augmented memory generated asynchronously by a Large Language Model (LLM) in the cloud. By integrating this retrieval augmented memory, the client model acquires the capability to generate highly effective responses, benefiting from the LLM's capabilities. Furthermore, through asynchronous memory integration, the client model is capable of delivering real-time responses to user requests without the need to wait for memory synchronization from the cloud. Our experiments on Wikitext and Pile subsets show that HybridRAG achieves lower latency than a cloud-based retrieval-augmented LLM, while outperforming client-only models in utility.