In-Context Exemplars as Clues to Retrieving from Large Associative Memory

In recent years, large language models (LLMs) have garnered significant attention due to their ability to revolutionize natural language processing (NLP) by demonstrating impressive language understanding and reasoning capabilities (7; 6; 45; 56; 44). LLMs are first pretrained on extensive data using the language modeling technique where the model predicts the next token given a context. Without finetuning on task-specific data, LLMs leverage in-context learning (ICL), also referred to as few-shot prompting, to make predictions. Through ICL, LLMs can find underlying patterns of the input query through given in-context exemplars, such as a set of input/output pairs, and use them to complete the response. However, the effects of in-context exemplars on downstream performance via ICL and guidelines for formulating those exemplars (e.g., how to select exemplars and how many exemplars to use) remain unclear.