When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits
Magomere, Jabez, La Malfa, Emanuele, Tonneau, Manuel, Kazemi, Ashkan, Hale, Scott
–arXiv.org Artificial Intelligence
Online misinformation remains a critical challenge, and fact-checkers increasingly rely on embedding-based methods to retrieve relevant fact-checks. Yet, when debunked claims reappear in edited forms, the performance of these methods is unclear. In this work, we introduce a taxonomy of six common real-world misinformation edits and propose a perturbation framework that generates valid, natural claim variations. Our multi-stage retrieval evaluation reveals that standard embedding models struggle with user-introduced edits, while LLM-distilled embeddings offer improved robustness at a higher computational cost. Although a strong reranker helps mitigate some issues, it cannot fully compensate for first-stage retrieval gaps. Addressing these retrieval gaps, our train- and inference-time mitigation approaches enhance in-domain robustness by up to 17 percentage points and boost out-of-domain generalization by 10 percentage points over baseline models. Overall, our findings provide practical improvements to claim-matching systems, enabling more reliable fact-checking of evolving misinformation.
arXiv.org Artificial Intelligence
Mar-6-2025
- Country:
- North America
- United States
- New York (0.04)
- Washington > King County
- Seattle (0.04)
- Minnesota > Hennepin County
- Minneapolis (0.14)
- Mexico > Mexico City
- Mexico City (0.04)
- Canada
- Ontario > Toronto (0.04)
- British Columbia > Metro Vancouver Regional District
- Vancouver (0.04)
- United States
- Europe
- United Kingdom > England
- Oxfordshire > Oxford (0.04)
- Middle East > Malta
- Eastern Region > Northern Harbour District > St. Julian's (0.04)
- Italy
- Ireland > Leinster
- County Dublin > Dublin (0.04)
- Greece > Central Macedonia
- Thessaloniki (0.04)
- United Kingdom > England
- Asia
- Singapore (0.04)
- China > Hong Kong (0.04)
- Indonesia > Bali (0.04)
- Thailand > Bangkok
- Bangkok (0.04)
- Myanmar > Tanintharyi Region
- Dawei (0.04)
- Middle East
- UAE > Abu Dhabi Emirate
- Abu Dhabi (0.04)
- Iran > Tehran Province
- Tehran (0.04)
- UAE > Abu Dhabi Emirate
- North America
- Genre:
- Research Report > New Finding (1.00)
- Industry:
- Media > News (1.00)
- Information Technology (1.00)
- Health & Medicine (1.00)
- Government > Regional Government
- Technology: