Demonstration Selection for In-Context Learning via Reinforcement Learning

Abstract--Diversity in demonstration selection is crucial for enhancing model generalization, as it enables a broader coverage of structures and concepts. However, constructing an appropriate set of demonstrations has remained a focal point of research. This paper presents the Relevance-Diversity Enhanced Selection (RDES), an innovative approach that leverages reinforcement learning to optimize the selection of diverse reference demonstrations for text classification tasks using Large Language Models (LLMs), especially in few-shot prompting scenarios. RDES employs a Q-learning framework to dynamically identify demonstrations that maximize both diversity and relevance to the classification objective by calculating a diversity score based on label distribution among selected demonstrations. This method ensures a balanced representation of reference data, leading to improved classification accuracy. This methodology allows LLMs to leverage their inherent LLMs have demonstrated exceptional capabilities across capabilities for understanding and processing text, making a wide array of NLP tasks, including text annotation [1], them particularly suitable for tasks with limited labeled data. These However, the effectiveness of ICL is contingent upon the models leverage extensive corpora of textual data to learn rich selection of appropriate and representative demonstrations representations, which empower them to perform reasoning from the knowledge base to serve as contextual references with high accuracy [4]-[6]. This critical aspect of fewshot of these models continue to expand, enhancing their learning is often overlooked in existing literature [12], reasoning capabilities becomes increasingly crucial.