Media
Studying Strategically: Learning to Mask for Closed-book QA
Ye, Qinyuan, Li, Belinda Z., Wang, Sinong, Bolte, Benjamin, Ma, Hao, Yih, Wen-tau, Ren, Xiang, Khabsa, Madian
Closed-book question-answering (QA) is a challenging task that requires a model to directly answer questions without access to external knowledge. It has been shown that directly fine-tuning pre-trained language models with (question, answer) examples yields surprisingly competitive performance, which is further improved upon through adding an intermediate pre-training stage between general pre-training and fine-tuning. Prior work used a heuristic during this intermediate stage, whereby named entities and dates are masked, and the model is trained to recover these tokens. In this paper, we aim to learn the optimal masking strategy for the intermediate pre-training stage. We first train our masking policy to extract spans that are likely to be tested, using supervision from the downstream task itself, then deploy the learned policy during intermediate pre-training. Thus, our policy packs task-relevant knowledge into the parameters of a language model. Our approach is particularly effective on TriviaQA, outperforming strong heuristics when used to pre-train BART.
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Min, Sewon, Boyd-Graber, Jordan, Alberti, Chris, Chen, Danqi, Choi, Eunsol, Collins, Michael, Guu, Kelvin, Hajishirzi, Hannaneh, Lee, Kenton, Palomaki, Jennimaria, Raffel, Colin, Roberts, Adam, Kwiatkowski, Tom, Lewis, Patrick, Wu, Yuxiang, Küttler, Heinrich, Liu, Linqing, Minervini, Pasquale, Stenetorp, Pontus, Riedel, Sebastian, Yang, Sohee, Seo, Minjoon, Izacard, Gautier, Petroni, Fabio, Hosseini, Lucas, De Cao, Nicola, Grave, Edouard, Yamada, Ikuya, Shimaoka, Sonse, Suzuki, Masatoshi, Miyawaki, Shumpei, Sato, Shun, Takahashi, Ryo, Suzuki, Jun, Fajcik, Martin, Docekal, Martin, Ondrej, Karel, Smrz, Pavel, Cheng, Hao, Shen, Yelong, Liu, Xiaodong, He, Pengcheng, Chen, Weizhu, Gao, Jianfeng, Oguz, Barlas, Chen, Xilun, Karpukhin, Vladimir, Peshterliev, Stan, Okhonko, Dmytro, Schlichtkrull, Michael, Gupta, Sonal, Mehdad, Yashar, Yih, Wen-tau
We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and return natural language answers. The aim of the competition was to build systems that can predict correct answers while also satisfying strict on-disk memory budgets. These memory budgets were designed to encourage contestants to explore the trade-off between storing large, redundant, retrieval corpora or the parameters of large learned models. In this report, we describe the motivation and organization of the competition, review the best submissions, and analyze system predictions to inform a discussion of evaluation for open-domain QA.
Factual Error Correction of Claims
Thorne, James, Vlachos, Andreas
This paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is supported by evidence. This serves two purposes: firstly this provides a mechanism to correct written texts that contain misinformation, and secondly, this acts as an inherent explanation for claims already partially supported by evidence. We demonstrate that factual error correction is possible without the need for any additional training data using distant-supervision and retrieved evidence. We release a dataset of 65,000 instances, based on a recent fact verification dataset, to compare our distantly-supervised method to a fully supervised ceiling system. Our manual evaluation indicates which automated evaluation metrics best correlate with human judgements of factuality and whether errors were actually corrected.
Vizio's affordable new soundbar trades excitement for simplicity
It should come as no surprise that Vizio infused the AIO with stellar sound for a reasonable price. From the V Series 2.1 to the company's impressive Dolby Atmos bars like the SB36512-F6, Vizio has built quite a reputation for packing its audio products with exceptional sound without reflecting it in the bars' retail price. With a six-driver array, simplistic equalizer options, and capable DTS Virtual:X, the AIO is well-equipped in the sound department among products in its price range. As inevitably tends to be the case, however, the savings come at a cost. The dual "subwoofers" built into the bar may be convincing for casual music listening, but are exposed when movies and TV shows demand more of them.
The Morning After: Boston Dynamics' dancing robots are back
Today, we've got stories on Apple's "ultra" security measures, someone squeezing entire movies on floppy disks and a deep dive on the ways we might connect, without touch, in a post-pandemic world. But for this opening salvo, let's home in on a family of dancing robots. Watch the Atlas robot and the entire Boston Dynamics family, including the dog-like Spot and box-stacking Handle, dance to "Do You Love Me" from The Contours, and you'll either feel affection or, well, repulsion. Boston Dynamics may now be 80 percent owned by car maker Hyundai, but it's keeping its sense of humor. Despite Google's Home Max being officially retired and pulled from sale a couple of weeks ago, the Google Store is once again offering the speaker for sale.