Government
ADVISE: AI-accelerated Design of Evidence Synthesis for Global Development
Edwards, Kristen M., Song, Binyang, Porciello, Jaron, Engelbert, Mark, Huang, Carolyn, Ahmed, Faez
When designing evidence-based policies and programs, decision-makers must distill key information from a vast and rapidly growing literature base. Identifying relevant literature from raw search results is time and resource intensive, and is often done by manual screening. In this study, we develop an AI agent based on a bidirectional encoder representations from transformers (BERT) model and incorporate it into a human team designing an evidence synthesis product for global development. We explore the effectiveness of the human-AI hybrid team in accelerating the evidence synthesis process. To further improve team efficiency, we enhance the human-AI hybrid team through active learning (AL). Specifically, we explore different sampling strategies, including random sampling, least confidence (LC) sampling, and highest priority (HP) sampling, to study their influence on the collaborative screening process. Results show that incorporating the BERT-based AI agent into the human team can reduce the human screening effort by 68.5% compared to the case of no AI assistance and by 16.8% compared to the case of using a support vector machine (SVM)-based AI agent for identifying 80% of all relevant documents. When we apply the HP sampling strategy for AL, the human screening effort can be reduced even more: by 78.3% for identifying 80% of all relevant documents compared to no AI assistance. We apply the AL-enhanced human-AI hybrid teaming workflow in the design process of three evidence gap maps (EGMs) for USAID and find it to be highly effective. These findings demonstrate how AI can accelerate the development of evidence synthesis products and promote timely evidence-based decision making in global development in a human-AI hybrid teaming context.
Evaluating Adversarial Robustness on Document Image Classification
Fronteau, Timothรฉe, Paran, Arnaud, Shabou, Aymen
Adversarial attacks and defenses have gained increasing interest on computer vision systems in recent years, but as of today, most investigations are limited to natural images. However, many artificial intelligence models actually handle documentary data, which is very different from real world images. Hence, in this work, we try to apply the adversarial attack philosophy on documentary data and to protect models against such attacks. Our methodology is to implement untargeted gradient-based, transfer-based and score-based attacks and evaluate the impact of defenses such as adversarial training, JPEG input compression and grey-scale input transformation on the robustness of ResNet50 and EfficientNetB0 model architectures. To the best of our knowledge, no such work has been conducted by the community in order to study the impact of these attacks on the document image classification task.
HQP: A Human-Annotated Dataset for Detecting Online Propaganda
Maarouf, Abdurahman, Bรคr, Dominik, Geissler, Dominique, Feuerriegel, Stefan
Online propaganda poses a severe threat to the integrity of societies. However, existing datasets for detecting online propaganda have a key limitation: they were annotated using weak labels that can be noisy and even incorrect. To address this limitation, our work makes the following contributions: (1) We present HQP: a novel dataset (N=30,000) for detecting online propaganda with high-quality labels. To the best of our knowledge, HQP is the first dataset for detecting online propaganda that was created through human annotation. (2) We show empirically that state-of-the-art language models fail in detecting online propaganda when trained with weak labels (AUC: 64.03). In contrast, state-of-the-art language models can accurately detect online propaganda when trained with our high-quality labels (AUC: 92.25), which is an improvement of ~44%. (3) To address the cost of labeling, we extend our work to few-shot learning. Specifically, we show that prompt-based learning using a small sample of high-quality labels can still achieve a reasonable performance (AUC: 80.27). Finally, we discuss implications for the NLP community to balance the cost and quality of labeling. Crucially, our work highlights the importance of high-quality labels for sensitive NLP tasks such as propaganda detection.
A Justice-Based Framework for the Analysis of Algorithmic Fairness-Utility Trade-Offs
Hertweck, Corinna, Baumann, Joachim, Loi, Michele, Viganรฒ, Eleonora, Heitz, Christoph
In prediction-based decision-making systems, different perspectives can be at odds: The short-term business goals of the decision makers are often in conflict with the decision subjects' wish to be treated fairly. Balancing these two perspectives is a question of values. However, these values are often hidden in the technicalities of the implementation of the decision-making system. In this paper, we propose a framework to make these value-laden choices clearly visible. We focus on a setting in which we want to find decision rules that balance the perspective of the decision maker and of the decision subjects. We provide an approach to formalize both perspectives, i.e., to assess the utility of the decision maker and the fairness towards the decision subjects. In both cases, the idea is to elicit values from decision makers and decision subjects that are then turned into something measurable. For the fairness evaluation, we build on well-known theories of distributive justice and on the algorithmic literature to ask what a fair distribution of utility (or welfare) looks like. This allows us to derive a fairness score that we then compare to the decision maker's utility. As we focus on a setting in which we are given a trained model and have to choose a decision rule, we use the concept of Pareto efficiency to compare decision rules. Our proposed framework can both guide the implementation of a decision-making system and help with audits, as it allows us to resurface the values implemented in a decision-making system.
Hitting the Books: Who's excited to have their brainwaves scanned as a personal ID?
All of those fantastical possibilities promised by burgeoning brain-computer interface technology come with the unavoidable cost of needing its potentially hackable wetware to ride shotgun in your skull. Given how often our personal data is already mishandled online, do we really want to trust the Tech Bros of Silicon Valley with our most personal of biometrics, our brainwaves? In her new book, The Battle for Your Brain: Defending the Right to Think Freely in the Age of Neurotechnology, Robinson O. Everett Professor of Law at Duke University, Nita A. Farahany, examines the legal, ethical, and moral threats that tomorrow's neurotechnologies could pose. From The Battle for Your Brain: Defending the Right to Think Freely in the Age of Neurotechnology by Nita A. Farahany. Assume that Meta, Google, Microsoft, and other big tech companies soon have their way, and neural interface devices replace keyboards and mice.
How Washington can help our moonshot against cancer become a reality
Almost every single one of us is impacted by cancer. For one of us, it happened right after taking office as governor of Maryland. Just six months into the job of a lifetime, I was diagnosed with a very advanced and very aggressive form of cancer. As I took on this challenge and governed my state from a hospital bed, I had the fortune of meeting so many incredible people who were going through much tougher battles than mine. For the other one of us, cancer struck when it took the life of my best friend and former chief of staff far too soon.
AI chatbot's 'bedside manner' preferred over conventional doctors by shocking margin, according to blind study
Chris Winfield, founder of Understanding A.I., tells Fox & Friends Weekend host Will Cain about a study showing patients preferred medical answers from artificial intelligence over doctors. Patients are becoming more favorable to having artificial intelligence involved in medicine, according to one study from The Journal of American Medicine, showing that nearly 80% of participants preferred a chatbot's medical responses over a conventional doctor's. "They liked the bedside manner of the A.I. doctor, in this case it was ChatGPT, better than the actual doctors themselves, and they actually felt more comfortable with those answers," said Chris Winfield, founder of Understanding A.I. Doctor surgeon and neurologist use robotic and medical technology to diagnose and examine patient brain with intelligence software. Winfield, who appeared Sunday on "Fox & Friends Weekend," said the blind study kept participants in the dark about who โ or what โ offered advice for their questions to more accurately shirk off potential biases. He added that one of the implications is that people are unhappy with conventional doctors' bedside manner.
Nearly 70% call for regulating development of AI bots, survey finds
The Japanese public harbors concerns about the rapidly spreading use of AI chatbots, with 69.4% calling for stricter regulation on the development of artificial intelligence, a Kyodo News poll showed Sunday. The result comes as countries have been discussing the need for international standards to prevent the misuse of emerging technologies such as ChatGPT, which have sparked fear of unauthorized collection of personal data. The telephone survey, conducted from Saturday, also marked the approval rating for the Cabinet of Prime Minister Fumio Kishida at 46.6%, up 8.5 points from the previous survey in mid-March. Its disapproval rating stood at 35.5%. This could be due to a conflict with your ad-blocking or security software. Please add japantimes.co.jp and piano.io to your list of allowed sites.
Police using AI could lead to 'predictive' crime prevention 'slippery slope,' experts argue
Recording Industry Association of America CEO Mitch Glazier says the Human Artistry Campaign aims to protect professional creators' rights to their performances, voices and likenesses after AI creates Drake and The Weeknd songs. A pilot program in the U.K. to enhance police capabilities via artificial intelligence has proven successful but could pave the way for a slide into a future of "predictive policing," experts told Fox News Digital. "Artificial intelligence is a tool, like a firearm is a tool, and it can be useful, it can be deadly," Christopher Alexander, CCO of Liberty Blockchain, told Fox News Digital. "In terms of the Holy Grail here, I really think it is the predictive analytics capability that if they get better at that, you have some very frightening capabilities." British police in different communities have experimented with an artificial intelligence-powered (AI) system to help catch drivers committing violations, such as using their phones while driving or driving without a seat belt.
Qualitative Analysis of a Graph Transformer Approach to Addressing Hate Speech: Adapting to Dynamically Changing Content
Hebert, Liam, Chen, Hong Yi, Cohen, Robin, Golab, Lukasz
Our work advances an approach for predicting hate speech in social media, drawing out the critical need to consider the discussions that follow a post to successfully detect when hateful discourse may arise. Using graph transformer networks, coupled with modelling attention and BERT-level natural language processing, our approach can capture context and anticipate upcoming anti-social behaviour. In this paper, we offer a detailed qualitative analysis of this solution for hate speech detection in social networks, leading to insights into where the method has the most impressive outcomes in comparison with competitors and identifying scenarios where there are challenges to achieving ideal performance. Included is an exploration of the kinds of posts that permeate social media today, including the use of hateful images. This suggests avenues for extending our model to be more comprehensive. A key insight is that the focus on reasoning about the concept of context positions us well to be able to support multi-modal analysis of online posts. We conclude with a reflection on how the problem we are addressing relates especially well to the theme of dynamic change, a critical concern for all AI solutions for social impact. We also comment briefly on how mental health well-being can be advanced with our work, through curated content attuned to the extent of hate in posts.