Goto

Collaborating Authors

 augenstein


Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection

arXiv.org Machine Learning

Stance Detection is concerned with identifying the attitudes expressed by an author towards a target of interest. This task spans a variety of domains ranging from social media opinion identification to detecting the stance for a legal claim. However, the framing of the task varies within these domains, in terms of the data collection protocol, the label dictionary and the number of available annotations. Furthermore, these stance annotations are significantly imbalanced on a per-topic and inter-topic basis. These make multi-domain stance detection a challenging task, requiring standardization and domain adaptation. To overcome this challenge, we propose $\textbf{T}$opic $\textbf{E}$fficient $\textbf{St}$anc$\textbf{E}$ $\textbf{D}$etection (TESTED), consisting of a topic-guided diversity sampling technique and a contrastive objective that is used for fine-tuning a stance classifier. We evaluate the method on an existing benchmark of $16$ datasets with in-domain, i.e. all topics seen and out-of-domain, i.e. unseen topics, experiments. The results show that our method outperforms the state-of-the-art with an average of $3.5$ F1 points increase in-domain, and is more generalizable with an averaged increase of $10.2$ F1 on out-of-domain evaluation while using $\leq10\%$ of the training data. We show that our sampling technique mitigates both inter- and per-topic class imbalances. Finally, our analysis demonstrates that the contrastive learning objective allows the model a more pronounced segmentation of samples with varying labels.


Measuring Intersectional Biases in Historical Documents

arXiv.org Artificial Intelligence

Data-driven analyses of biases in historical texts can help illuminate the origin and development of biases prevailing in modern society. However, digitised historical documents pose a challenge for NLP practitioners as these corpora suffer from errors introduced by optical character recognition (OCR) and are written in an archaic language. In this paper, we investigate the continuities and transformations of bias in historical newspapers published in the Caribbean during the colonial era (18th to 19th centuries). Our analyses are performed along the axes of gender, race, and their intersection. We examine these biases by conducting a temporal study in which we measure the development of lexical associations using distributional semantics models and word embeddings. Further, we evaluate the effectiveness of techniques designed to process OCR-generated data and assess their stability when trained on and applied to the noisy historical newspapers. We find that there is a trade-off between the stability of the word embeddings and their compatibility with the historical dataset. We provide evidence that gender and racial biases are interdependent, and their intersection triggers distinct effects. These findings align with the theory of intersectionality, which stresses that biases affecting people with multiple marginalised identities compound to more than the sum of their constituents.


Modeling Information Change in Science Communication with Semantically Matched Paraphrases

arXiv.org Artificial Intelligence

Whether the media faithfully communicate scientific information has long been a core issue to the science community. Automatically identifying paraphrased scientific findings could enable large-scale tracking and analysis of information changes in the science communication process, but this requires systems to understand the similarity between scientific information across multiple domains. To this end, we present the SCIENTIFIC PARAPHRASE AND INFORMATION CHANGE DATASET (SPICED), the first paraphrase dataset of scientific findings annotated for degree of information change. SPICED contains 6,000 scientific finding pairs extracted from news stories, social media discussions, and full texts of original papers. We demonstrate that SPICED poses a challenging task and that models trained on SPICED improve downstream performance on evidence retrieval for fact checking of real-world scientific claims. Finally, we show that models trained on SPICED can reveal large-scale trends in the degrees to which people and organizations faithfully communicate new scientific findings. Data, code, and pre-trained models are available at http://www.copenlu.com/publication/2022_emnlp_wright/.


Augenstein

AAAI Conferences

A mixed-integer linear program (MILP) approach to scheduling a large constellation of Earth-imaging satellites is presented. The algorithm optimizes the assignment of imagery collects, image data downlinks, and "health & safety" contacts, generating schedules for all satellites and ground stations in a network. Hardware-driven constraints (e.g., the limited agility of the satellites) and operations-driven constraints (e.g., guaranteeing a minimum contact frequency for each satellite) are both addressed. Of critical importance to the use of this algorithm in real-world operations, it runs fast enough to allow for human operator interaction and repeated rescheduling. This is achieved by a partitioning of the problem into sequential steps for downlink scheduling and image scheduling, with a novel dynamic programming (DP) heuristic providing a stand-in for imaging activity in the MILP when scheduling the downlinks.


Does Typological Blinding Impede Cross-Lingual Sharing?

arXiv.org Artificial Intelligence

Bridging the performance gap between high- and low-resource languages has been the focus of much previous work. Typological features from databases such as the World Atlas of Language Structures (WALS) are a prime candidate for this, as such data exists even for very low-resource languages. However, previous work has only found minor benefits from using typological information. Our hypothesis is that a model trained in a cross-lingual setting will pick up on typological cues from the input data, thus overshadowing the utility of explicitly using such features. We verify this hypothesis by blinding a model to typological information, and investigate how cross-lingual sharing and performance is impacted. Our model is based on a cross-lingual architecture in which the latent weights governing the sharing between languages is learnt during training. We show that (i) preventing this model from exploiting typology severely reduces performance, while a control experiment reaffirms that (ii) encouraging sharing according to typology somewhat improves performance.


What reading 3.5 million books tells us about gender stereotypes

#artificialintelligence

Huge social questions like "how are men and women perceived differently" cannot be easily answered without analyzing rhetoric on a massive scale. But what if we could analyze millions of words, all at once, to get a sense of what patterns emerge in how men and women were described? It wasn't until recently that machine learning algorithms could help researchers do just that. In a recent study, Dr. Isabelle Augenstein, a computer scientist at the University of Copenhagen, worked with fellow researchers from the United States to analyze 11 billion words in an effort to find out whether there was a difference between the adjectives used to describe men and women in literature. The researchers examined a dataset of 3.5 million books, all published in English between 1900 to 2008.


Massive Machine Learning Study Demonstrates Gender Stereotyping And Sexist Language In Literature

#artificialintelligence

An unsupervised machine learning study presented at the 2019 meeting of Association for Computational Linguistics--which examined 3.5M books published between 1900 and 2008--indicates that men are described based on their behavior, where women are described based on appearance. In specific, words like "beautiful" and "sexy" are two of the adjectives most frequently used to describe women, while common descriptors for men were "brave," "rational," and "righteous." The books, which amounted to approximately 11B words in sum, included a mix of fiction and non-fiction. "We are clearly able to see that the words used for women refer much more to their appearances than the words used to describe men," said University of Copenhagen computer scientist and assistant professor Isabelle Augenstein in a statement. "Thus, we have been able to confirm a widespread perception, only now at a statistical level."


Massive Machine Learning Study Demonstrates Gender Stereotyping And Sexist Language In Literature

#artificialintelligence

An unsupervised machine learning study presented at the 2019 meeting of Association for Computational Linguistics--which examined 3.5M books published between 1900 and 2008--indicates that men are described based on their behavior, where women are described based on appearance. In specific, words like "beautiful" and "sexy" are two of the adjectives most frequently used to describe women, while common descriptors for men were "brave," "rational," and "righteous." The books, which amounted to approximately 11B words in sum, included a mix of fiction and non-fiction. "We are clearly able to see that the words used for women refer much more to their appearances than the words used to describe men," said University of Copenhagen computer scientist and assistant professor Isabelle Augenstein in a statement. "Thus, we have been able to confirm a widespread perception, only now at a statistical level."


Algorithms find top 11 adjectives for men v. women in 3.5M books - Futurity

#artificialintelligence

You are free to share this article under the Attribution 4.0 International license. Machine learning analyzed 3.5 million books to find that adjectives ascribed to women tend to describe physical appearance, whereas words that refer to behavior go to men. "Beautiful" and "sexy" are two of the adjectives most frequently used to describe women. Commonly used descriptors for men include righteous, rational, and courageous. Researchers trawled through an enormous quantity of books in an effort to find out whether there is a difference between the types of words that describe men and women in literature.


If AI Can Fix Peer Review in Science, AI Can Do Anything

WIRED

Here's how science works: You have a question about some infinitesimal sliver of the universe. You form a hypothesis, test it, and eventually gather enough data to support or disprove what you thought was going on. The next bit is less glamorous: You write a manuscript, submit it to an academic journal, and endure the gauntlet of peer review, where a small group of anonymous experts in your field scrutinize the quality of your work. Peer review has its flaws. Human beings (even scientists) are biased, lazy, and self-interested.