We present GraPPa, an effective pre-training approach for table semantic parsing that learns a compositional inductive bias in the joint representations of textual and tabular data. We construct synthetic question-SQL pairs over high-quality tables via a synchronous context-free grammar (SCFG) induced from existing text-to-SQL datasets. We pre-train our model on the synthetic data using a novel text-schema linking objective that predicts the syntactic role of a table field in the SQL for each question-SQL pair. To maintain the model's ability to represent real-world data, we also include masked language modeling (MLM) over several existing table-and-language datasets to regularize the pre-training process. On four popular fully supervised and weakly supervised table semantic parsing benchmarks, GraPPa significantly outperforms RoBERTa-large as the feature representation layers and establishes new state-of-the-art results on all of them.

There has rarely been as much excitement in the phone world as over news that the iconic Nokia 3310 is making a comeback. Launched way back in 2000, a naive age when people bought mobile phones in order to talk to each other, the handset is still famed for its lengthy battery life, structural solidity and Snake II. Seventeen years later, modern smartphones are crammed with high-tech features, but you have to charge them constantly and their demands on our attention – via endless social media alerts, updates and notifications – are becoming tiresome. Some people yearn for a simpler age when the phone just did what it was primarily designed for and most of your text messages were from confused relatives saying'AM I USING THIS CORRECTLY'. This kind of industrial technology nostalgia is usually just that – nostalgia.