MT-RAIG: Novel Benchmark and Evaluation Framework for Retrieval-Augmented Insight Generation over Multiple Tables
Seo, Kwangwook, Kwon, Donguk, Lee, Dongha
–arXiv.org Artificial Intelligence
Recent advancements in table-based reasoning have expanded beyond factoid-level QA to address insight-level tasks, where systems should synthesize implicit knowledge in the table to provide explainable analyses. Although effective, existing studies remain confined to scenarios where a single gold table is given alongside the user query, failing to address cases where users seek comprehensive insights from multiple unknown tables. To bridge these gaps, we propose MT-RAIG Bench, design to evaluate systems on Retrieval-Augmented Insight Generation over Mulitple-Tables. Additionally, to tackle the suboptimality of existing automatic evaluation methods in the table domain, we further introduce a fine-grained evaluation framework MT-RAIG Eval, which achieves better alignment with human quality judgments on the generated insights. We conduct extensive experiments and reveal that even frontier LLMs still struggle with complex multi-table reasoning, establishing our MT-RAIG Bench as a challenging testbed for future research.
arXiv.org Artificial Intelligence
Feb-18-2025
- Country:
- Asia
- Armenia > Yerevan
- Yerevan (0.04)
- China
- Kazakhstan > Qaraghandy Region
- Karagandy (0.04)
- Middle East > UAE
- Dubai Emirate > Dubai (0.04)
- Myanmar > Tanintharyi Region
- Dawei (0.04)
- Pakistan
- Islamabad Capital Territory > Islamabad (0.04)
- Punjab > Lahore Division
- Lahore (0.04)
- Sindh > Karachi Division
- Karachi (0.04)
- Russia (0.04)
- Singapore (0.04)
- Armenia > Yerevan
- Europe
- Belarus
- Gomel Region > Gomel (0.04)
- Minsk Region > Minsk (0.05)
- Czechia > Prague (0.04)
- Finland > Uusimaa
- Helsinki (0.04)
- Latvia
- Liepāja Municipality > Liepāja (0.04)
- Riga Municipality > Riga (0.04)
- Middle East > Cyprus
- Russia
- France (0.04)
- Germany (0.04)
- Slovakia > Bratislava
- Bratislava (0.04)
- Austria > Styria
- Graz (0.04)
- Poland > Greater Poland Province
- Poznań (0.04)
- Spain > Galicia
- Madrid (0.04)
- Belarus
- North America
- Mexico > Mexico City
- Mexico City (0.04)
- United States
- Florida > Miami-Dade County
- Miami (0.04)
- Minnesota (0.04)
- New York > New York County
- New York City (0.04)
- Virginia (0.04)
- Florida > Miami-Dade County
- Mexico > Mexico City
- Oceania > Australia (0.04)
- Asia
- Genre:
- Research Report > New Finding (0.68)
- Industry:
- Automobiles & Trucks > Manufacturer (1.00)
- Leisure & Entertainment > Sports
- Soccer (0.94)
- Media > Television (0.92)
- Technology: