SING-SQL: A Synthetic Data Generation Framework for In-Domain Text-to-SQL Translation
Caferoğlu, Hasan Alp, Çelik, Mehmet Serhat, Ulusoy, Özgür
–arXiv.org Artificial Intelligence
Translating natural language questions into SQL has become a core challenge in enabling non-technical users to query databases. While recent work has explored large-scale synthetic data generation to improve model performance through post-training, most efforts emphasize cross-domain generalization. This leaves a gap for real-world enterprise scenarios, where models need to specialize to a single database schema and organizations require to be able to evaluate their Text-to-SQL systems on their own databases. To address this, we introduce SING-SQL, a fully automated two-stage framework for generating high-quality, high-coverage synthetic Text-to-SQL data for any target database, without relying on SQL logs or manual annotations. Our approach hierarchically partitions a database schema into sub-schemas, synthesizes SQL queries across multiple complexity levels, and applies a quality-aware pipeline that includes LLM-as-a-judge validation, executability checks, automatic repair, and column balancing. We further release SingSQL-LM, a family of compact language models fine-tuned on the synthetic data, achieving strong in-domain generalization. On the subset of the BIRD benchmark, SingSQL-LM-3B-R64 reaches 82.87% Soft F1 and 73.03% EX upper bound with 32 candidates, outperforming the best 3B-scale baseline by +16.21 in Soft F1 and +12.36 in EX. At the 1.5B scale, SingSQL-LM-1.5B-R64 improves over prior systems by +9.30 in Soft F1 and +4.49 in EX. On synthetic evaluation sets, SingSQL-LMs exceed prior systems by wide margins, establishing state-of-the-art performance among open models at comparable scales. Our study of context management strategies reveals that schema-free fine-tuning combined with schema-only inference provides the most robust results. These findings establish SING-SQL as a scalable, database-agnostic paradigm for producing and evaluating enterprise-grade Text-to-SQL systems.
arXiv.org Artificial Intelligence
Oct-1-2025
- Country:
- Asia
- Middle East
- Republic of Türkiye > Ankara Province
- Ankara (0.04)
- UAE > Abu Dhabi Emirate
- Abu Dhabi (0.14)
- Republic of Türkiye > Ankara Province
- Myanmar > Tanintharyi Region
- Dawei (0.04)
- Thailand > Bangkok
- Bangkok (0.04)
- Middle East
- Europe > Austria
- Vienna (0.14)
- North America > United States
- California > Los Angeles County
- Los Angeles (0.04)
- Louisiana > Orleans Parish
- New Orleans (0.04)
- New York > New York County
- New York City (0.04)
- California > Los Angeles County
- Asia
- Genre:
- Research Report > New Finding (0.67)
- Industry:
- Education (1.00)
- Technology: