Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks

In recent years, the use of large language models (LLMs) has significantly increased, and these models have demonstrated remarkable performance in a variety of general language tasks. However, the evaluation of their performance in domain-specific tasks, particularly those requiring deep natural language understanding, has received less attention. In this research, we evaluate the ability of large language models in performing domain-specific tasks, focusing on the multi-hop question answering (MHQA) problem using the HotpotQA dataset. This task, due to its requirement for reasoning and combining information from multiple textual sources, serves as a challenging benchmark for assessing the language comprehension capabilities of these models. To tackle this problem, we have designed a two-stage selector-reader architecture, where each stage utilizes an independent LLM. In addition, methods such as Chain of Thought (CoT) and question decomposition have been employed to investigate their impact on improving the model's performance. The results of the study show that the integration of large language models with these techniques can lead to up to a 4% improvement in F1 score for finding answers, providing evidence of the models' ability to handle domain-specific tasks and their understanding of complex language. Keywords Large Language Models, Multi-hop Question Answering, Task Decomposition, Knowledge Distillation, HotpotQA Introduction In recent years, LLMs have become one of the most significant achievements in natural language processing, demonstrating exceptional performance in a wide range of general language tasks, such as translation, summarization, and text generation. However, existing evaluations are often based on general benchmarks and employ zero-shot or few-shot approaches. While these methods showcase the overall capabilities of the models, they do not provide a thorough and in-depth analysis of their performance on specific tasks. In other words, these evaluations do not compare the models' capabilities to traditional methods or models optimized for a particular task. Therefore, investigating how to improve the performance of language models on domain-specific tasks and identifying their limitations remains an important and underexplored research gap.