Do state-of-the-art natural language understanding models care about word order - one of the most important characteristics of a sequence? Not always! We found 75% to 90% of the correct predictions of BERT-based classifiers, trained on many GLUE tasks, remain constant after input words are randomly shuffled. Despite BERT embeddings are famously contextual, the contribution of each individual word to downstream tasks is almost unchanged even after the word's context is shuffled. BERT-based models are able to exploit superficial cues (e.g. the sentiment of keywords in sentiment analysis; or the word-wise similarity between sequence-pair inputs in natural language inference) to make correct decisions when tokens are arranged in random orders. Encouraging classifiers to capture word order information improves the performance on most GLUE tasks, SQuAD 2.0 and out-of-samples. Our work suggests that many GLUE tasks are not challenging machines to understand the meaning of a sentence.

However, while the film is light-hearted, affectionate and comedic in tone, it carries a very important and timely message about trust and the ease with which convincing misinformation can be created and spread.

Audeze is one of the only headphone makers devoted exclusively to using planar-magnetic (PM) drivers in its products. The company offers a wide range of models, from reference over-ear designs to gaming-specific cans to in-ear monitors, all of which are based on PM technology. I have little experience with planar-magnetic headphones. Many years ago, I reviewed the Stax SR-007 MK2 electrostatic headphones, which use a somewhat similar technology, but they require a special amplifier and are extremely expensive. But I've never spent any quality time with PM headphones.

Open-domain dialogue agents have vastly improved, but still confidently hallucinate knowledge or express doubt when asked straightforward questions. In this work, we analyze whether state-of-the-art chit-chat models can express metacognition capabilities through their responses: does a verbalized expression of doubt (or confidence) match the likelihood that the model's answer is incorrect (or correct)? We find that these models are poorly calibrated in this sense, yet we show that the representations within the models can be used to accurately predict likelihood of correctness. By incorporating these correctness predictions into the training of a controllable generation model, we obtain a dialogue agent with greatly improved linguistic calibration.

We introduce twin neural network (TNN) regression. This method predicts differences between the target values of two different data points rather than the targets themselves. The solution of a traditional regression problem is then obtained by averaging over an ensemble of all predicted differences between the targets of an unseen data point and all training data points. Whereas ensembles are normally costly to produce, TNN regression intrinsically creates an ensemble of predictions of twice the size of the training set while only training a single neural network. Since ensembles have been shown to be more accurate than single models this property naturally transfers to TNN regression. We show that TNNs are able to compete or yield more accurate predictions for different data sets, compared to other state-of-the-art methods. Furthermore, TNN regression is constrained by self-consistency conditions. We find that the violation of these conditions provides an estimate for the prediction uncertainty.

"This machine is capable of running games at top-level specs, all on top of keeping your meal warm for you to enjoy during your gaming experience... what's not to like?" KFC spokesman Mark Cheevers told the BBC.

Celebrities spread Covid conspiracies and video games were shared as ‘real’ war footage.

Fox News Flash top entertainment and celebrity headlines are here. Check out what's clicking today in entertainment. Lin Qi, an executive producer on "Game of Thrones" creators David Benioff and D.B. Weiss' upcoming Netflix series, died at age 39 after being poisoned. Lin was the chairman and CEO of Yoozoo Group, which he founded in 2009. The company was working with the TV creators on an adaptation of a science fiction series based on "The Three-Body Problem" trilogy of novels by Chinese author Liu Cixin.

Fox News Flash top entertainment and celebrity headlines are here. Check out what's clicking today in entertainment. Britain's Channel 4 last week produced a stunningly real-looking parody video of Queen Elizabeth's annual Christmas Day message that the network claims highlights the dangers of "deepfake" technology. Channel 4 has been releasing its own "alternative" Christmas message for nearly 30 years and decided to make a deepfake video this year as a warning about the technology's potential dangers. The technique of manipulating someone's face and voice in a "deepfake" video is "more easy than most people would think," the channel said in a separate video showing how it synthetically recreated the queen with the help of actress Debra Stephenson.

Humans give contrastive explanations that explain why an observed event happened rather than some other counterfactual event (the contrast case). Despite the important role that contrastivity plays in how people generate and evaluate explanations, this property is largely missing from current methods for explaining NLP models. We present Minimal Contrastive Editing (MiCE), a method for generating contrastive explanations of model predictions in the form of edits to inputs that change model outputs to the contrast case. Our experiments across three tasks -- binary sentiment classification, topic classification, and multiple-choice question answering -- show that MiCE is able to produce edits that are not only contrastive, but also minimal and fluent, consistent with human contrastive edits. We demonstrate how MiCE edits can be used for two use cases in NLP system development -- uncovering dataset artifacts and debugging incorrect model predictions -- and thereby illustrate that generating contrastive explanations is a promising research direction for model interpretability.