Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documents

Methods for relation extraction from text mostly focus on high precision, at the cost of limited recall. High recall is crucial, though, to populate long lists of object entities that stand in a specific relation with a given subject. Cues for relevant objects can be spread across many passages in long texts. This poses the challenge of extracting long lists from long texts. We present the L3X method which tackles the problem in two stages: (1) recall-oriented generation using a large language model (LLM) with judicious techniques for retrieval augmentation, and (2) precision-oriented scrutinization to validate or prune candidates. Our L3X method outperforms Figure 1: Example for extracting long lists from long LLM-only generations by a substantial texts. For the subject "Harry Potter", we aim to extract margin.