Media
CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge
Onoe, Yasumasa, Zhang, Michael J. Q., Choi, Eunsol, Durrett, Greg
Most benchmark datasets targeting commonsense reasoning focus on everyday scenarios: physical knowledge like knowing that you could fill a cup under a waterfall [Talmor et al., 2019], social knowledge like bumping into someone is awkward [Sap et al., 2019], and other generic situations. However, there is a rich space of commonsense inferences anchored to knowledge about specific entities: for example, deciding the truthfulness of a claim "Harry Potter can teach classes on how to fly on a broomstick." Can models learn to combine entity knowledge with commonsense reasoning in this fashion? We introduce CREAK, a testbed for commonsense reasoning about entity knowledge, bridging fact-checking about entities (Harry Potter is a wizard and is skilled at riding a broomstick) with commonsense inferences (if you're good at a skill you can teach others how to do it). Our dataset consists of 13k human-authored English claims about entities that are either true or false, in addition to a small contrast set. Crowdworkers can easily come up with these statements and human performance on the dataset is high (high 90s); we argue that models should be able to blend entity knowledge and commonsense reasoning to do well here. In our experiments, we focus on the closed-book setting and observe that a baseline model finetuned on existing fact verification benchmark struggles on CREAK. Training a model on CREAK improves accuracy by a substantial margin, but still falls short of human performance. Our benchmark provides a unique probe into natural language understanding models, testing both its ability to retrieve facts (e.g., who teaches at the University of Chicago?) and unstated commonsense knowledge (e.g., butlers do not yell at guests).
Coordinating Narratives and the Capitol Riots on Parler
Ng, Lynnette Hui Xian, Cruickshank, Iain, Carley, Kathleen M.
Coordinated disinformation campaigns are used to influence social media users, potentially leading to offline violence. In this study, we introduce a general methodology to uncover coordinated messaging through analysis of user parleys on Parler. The proposed method constructs a user-to-user coordination network graph induced by a user-to-text graph and a text-to-text similarity graph. The text-to-text graph is constructed based on the textual similarity of Parler posts. We study three influential groups of users in the 6 January 2020 Capitol riots and detect networks of coordinated user clusters that are all posting similar textual content in support of different disinformation narratives related to the U.S. 2020 elections.
Challenges in Generalization in Open Domain Question Answering
Liu, Linqing, Lewis, Patrick, Riedel, Sebastian, Stenetorp, Pontus
Recent work on Open Domain Question Answering has shown that there is a large discrepancy in model performance between novel test questions and those that largely overlap with training questions. However, it is as of yet unclear which aspects of novel questions that make them challenging. Drawing upon studies on systematic generalization, we introduce and annotate questions according to three categories that measure different levels and kinds of generalization: training set overlap, compositional generalization (comp-gen), and novel entity generalization (novel-entity). When evaluating six popular parametric and non-parametric models, we find that for the established Natural Questions and TriviaQA datasets, even the strongest model performance for comp-gen/novel-entity is 13.1/5.4% and 9.6/1.5% lower compared to that for the full test set -- indicating the challenge posed by these types of questions. Furthermore, we show that whilst non-parametric models can handle questions containing novel entities, they struggle with those requiring compositional generalization. Through thorough analysis we find that key question difficulty factors are: cascading errors from the retrieval component, frequency of question pattern, and frequency of the entity.
Rage Against the Machine's Tom Morello asks for help in getting female guitar students out of Afghanistan
Fox News Flash top entertainment and celebrity headlines are here. Check out what's clicking today in entertainment. Rage Against the Machine's Tom Morello is asking for the public's help in assisting female guitar students out of Afghanistan following the Taliban takeover of the capital city, Kabul. Morello penned an open letter calling attention to his friend and musician Lanny Cordola's "Girl with a Guitar" program, which currently has 12 female guitar students ranging in age from 8 to 17 years old who are stuck in Kabul. The girls are students of a music school in Afghanistan Cordola helped bring to fruition through a non-profit called Miraculous Love Kids.
VOXOX Expands Expert Teams for a Future in Artificial Intelligence
VOXOX, a 5G-enabled AI cloud communications company, today announced the expansion of its UI/UX and data analytics teams for strategic growth. This step forward makes room for additional research and development of solutions to help automation become an essential and integral option for small businesses. VOXOX's engineering and data science teams have spent years analyzing billions of data points within the voice and text networks to help them develop artificial intelligence that can analyze content, respond like a real person, handle daily tasks like customer service inquiries & support jobs, schedule appointments, and write effective text campaigns. "The future of 5G AI telecommunications is at hand and we are eager to be growing rapidly in this field," said CEO of VOXOX, Cleve Adams. "VOXOX is leaping forward through the growth of our teams by bringing in skilled experts who have a great focus on providing remarkable automated solutions. Our intelligent assistant, virtual receptionists, and SMS drip campaigns are only the beginning of the AI-powered features that VOXOX has to offer."