Goto

Collaborating Authors

 Government


Explaining Mixtures of Sources in News Articles

arXiv.org Artificial Intelligence

Human writers plan, then write. For large language models (LLMs) to play a role in longer-form article generation, we must understand the planning steps humans make before writing. We explore one kind of planning, source-selection in news, as a case-study for evaluating plans in long-form generation. We ask: why do specific stories call for specific kinds of sources? We imagine a generative process for story writing where a source-selection schema is first selected by a journalist, and then sources are chosen based on categories in that schema. Learning the article's plan means predicting the schema initially chosen by the journalist. Working with professional journalists, we adapt five existing schemata and introduce three new ones to describe journalistic plans for the inclusion of sources in documents. Then, inspired by Bayesian latent-variable modeling, we develop metrics to select the most likely plan, or schema, underlying a story, which we use to compare schemata. We find that two schemata: stance and social affiliation best explain source plans in most documents. However, other schemata like textual entailment explain source plans in factually rich topics like "Science". Finally, we find we can predict the most suitable schema given just the article's headline with reasonable accuracy. We see this as an important case-study for human planning, and provides a framework and approach for evaluating other kinds of plans. We release a corpora, NewsSources, with annotations for 4M articles.


The Fundamental Rights Impact Assessment (FRIA) in the AI Act: Roots, legal obligations and key elements for a model template

arXiv.org Artificial Intelligence

What is the context which gave rise to the obligation to carry out a Fundamental Rights Impact Assessment (FRIA) in the AI Act? How has assessment of the impact on fundamental rights been framed by the EU legislator in the AI Act? What methodological criteria should be followed in developing the FRIA? These are the three main research questions that this article aims to address, through both legal analysis of the relevant provisions of the AI Act and discussion of various possible models for assessment of the impact of AI on fundamental rights. The overall objective of this article is to fill existing gaps in the theoretical and methodological elaboration of the FRIA, as outlined in the AI Act. In order to facilitate the future work of EU and national bodies and AI operators in placing this key tool for human-centric and trustworthy AI at the heart of the EU approach to AI design and development, this article outlines the main building blocks of a model template for the FRIA. While this proposal is consistent with the rationale and scope of the AI Act, it is also applicable beyond the cases listed in Article 27 and can serve as a blueprint for other national and international regulatory initiatives to ensure that AI is fully consistent with human rights.


GREI Data Repository AI Taxonomy

arXiv.org Artificial Intelligence

Authors: John Chodacki (California Digital Library), Mark Hanhel (figshare), Stefano Iacus (Dataverse), Ryan Scherle (Dryad), Eric Olson (Center for Open Science), Nici Pfeiffer (Center for Open Science), Kristi Holmes (Zenodo), Mohammad Hosseini (Zenodo) The Generalist Repository Ecosystem Initiative (GREI) is a NIH-funded program where repositories collaborate in a "coopetition" model to enhance the work of generalist data repositories, which are critical infrastructure across research domains. As part of our commitment to this work, we recognize the evolving importance of artificial intelligence (AI) in the future of science and infrastructure. To help our community navigate AI-driven changes, we have developed the following taxonomy to illustrate the roles AI can play in managing data repositories, improving data quality, and increasing accessibility. Building on previously developed taxonomies and our coopetition efforts, the GREI repositories propose the following "GREI Data Repository AI Taxonomy," specifically tailored for data repository roles. Why do we need this?


A Barrier Certificate-based Simplex Architecture for Systems with Approximate and Hybrid Dynamics

arXiv.org Artificial Intelligence

Bb-Simplex is centered around the Simplex control architecture, which consists of a high-performance advanced controller that is not guaranteed to maintain safety of the plant, a verified-safe baseline controller, and a decision module that switches control of the plant between the two controllers to ensure safety without sacrificing performance. In Bb-Simplex, Barrier certificates are used to prove that the baseline controller ensures safety. Furthermore, Bb-Simplex features a new automated method for deriving, from the barrier certificate, the conditions for switching between the controllers. Our method is based on the Taylor expansion of the barrier certificate and yields computationally inexpensive switching conditions. We also propose extensions to Bb-Simplex to enable its use in hybrid systems, which have multiple modes each with its own dynamics, and to support its use when only approximate dynamics (not exact dynamics) are available, for both continuous-time and hybrid dynamical systems. We consider significant applications of Bb-Simplex to microgrids featuring advanced controllers in the form of neural networks trained using reinforcement learning. These microgrids are modeled in RTDS, an industry-standard high-fidelity, real-time power systems simulator. Our results demonstrate that Bb-Simplex can automatically derive switching conditions for complex continuous-time and hybrid systems, the switching conditions are not overly conservative, and Bb-Simplex ensures safety even in the presence of adversarial attacks on the neural controller when only approximate dynamics (with an error bound) are available.


Soft-Label Integration for Robust Toxicity Classification

arXiv.org Artificial Intelligence

Toxicity classification in textual content remains a significant problem. Data with labels from a single annotator fall short of capturing the diversity of human perspectives. Therefore, there is a growing need to incorporate crowdsourced annotations for training an effective toxicity classifier. Additionally, the standard approach to training a classifier using empirical risk minimization (ERM) may fail to address the potential shifts between the training set and testing set due to exploiting spurious correlations. This work introduces a novel bi-level optimization framework that integrates crowdsourced annotations with the soft-labeling technique and optimizes the soft-label weights by Group Distributionally Robust Optimization (GroupDRO) to enhance the robustness against out-of-distribution (OOD) risk. We theoretically prove the convergence of our bi-level optimization algorithm. Experimental results demonstrate that our approach outperforms existing baseline methods in terms of both average and worst-group accuracy, confirming its effectiveness in leveraging crowdsourced annotations to achieve more effective and robust toxicity classification.


Findings of the IWSLT 2024 Evaluation Campaign

arXiv.org Artificial Intelligence

This paper reports on the shared tasks organized by the 21st IWSLT Conference. The shared tasks address 7 scientific challenges in spoken language translation: simultaneous and offline translation, automatic subtitling and dubbing, speech-to-speech translation, dialect and low-resource speech translation, and Indic languages. The shared tasks attracted 18 teams whose submissions are documented in 26 system papers. The growing interest towards spoken language translation is also witnessed by the constantly increasing number of shared task organizers and contributors to the overview paper, almost evenly distributed across industry and academia.


Selecting Between BERT and GPT for Text Classification in Political Science Research

arXiv.org Artificial Intelligence

Political scientists often grapple with data scarcity in text classification. Recently, fine-tuned BERT models and their variants have gained traction as effective solutions to address this issue. In this study, we investigate the potential of GPT-based models combined with prompt engineering as a viable alternative. We conduct a series of experiments across various classification tasks, differing in the number of classes and complexity, to evaluate the effectiveness of BERT-based versus GPT-based models in low-data scenarios. Our findings indicate that while zero-shot and few-shot learning with GPT models provide reasonable performance and are well-suited for early-stage research exploration, they generally fall short - or, at best, match - the performance of BERT fine-tuning, particularly as the training set reaches a substantial size (e.g., 1,000 samples). We conclude by comparing these approaches in terms of performance, ease of use, and cost, providing practical guidance for researchers facing data limitations. Our results are particularly relevant for those engaged in quantitative text analysis in low-resource settings or with limited labeled data.


Public Procurement for Responsible AI? Understanding U.S. Cities' Practices, Challenges, and Needs

arXiv.org Artificial Intelligence

Thus, most public-sector AI systems used today are developed by and acquired from private vendors. A growing number of academic and advocacy efforts have pointed out how AI systems procured in the public sector have predominantly targeted narrowly defined notions of efficiency and performance enhancements, resulting in adverse effects that disparately impact marginalized communities[18, 37, 46, 50, 86, 96]. While such incidents have exposed flaws in individual AI systems, they highlight deeper issues in how AI is acquired, used, and governed in the public sector. The AI procurement process encompasses decisions of which AI tools to ask for, adopt or reject, and the manner in which they are developed and deployed: decisions of critical importance for communities who may be harmed by AI. Such decisions not only influence the performance and risks posed by AI systems, but also play a significant role in shaping broader governance practices and ethical standards by which AI operates in the public sector. Interestingly, there is a long history of governments adapting their public procurement practices to enact social change, e.g., by creating processes that prioritize minority-owned businesses [62],


Sentiment Analysis of Spanish Political Party Tweets Using Pre-trained Language Models

arXiv.org Artificial Intelligence

Abstract: This study investigates sentiment patterns within Spanish political party communications on Twitter by employing BETO and RoBERTuito, two pre-trained language models optimized for Spanish text. With a dataset comprising tweets from major Spanish political parties--PSOE, PP, Vox, Podemos, and Ciudadanos--spanning 2019 to 2024, this research analyzes sentiment distributions and explores the relationship between sentiment and party ideology. Results reveal that both models consistently identify a predominant Neutral sentiment across parties, with significant variations in Negative and Positive sentiments that align with ideological distinctions. Vox exhibits higher levels of Negative sentiment, while PSOE demonstrates a relatively high Positive sentiment, supporting the hypothesis that emotional appeals in political messaging reflect ideological stances. This study highlights the utility of pre-trained models in analyzing non-English social media sentiment and underscores the implications of sentiment dynamics in shaping public discourse within a multi-party system. Keywords: Spanish political parties, sentiment analysis, Twitter, BETO, RoBERTuito, political communication, ideology, social media analysis 1. Introduction In the era of digital politics, social media has emerged as a potent platform where public opinion is actively shaped and reflected. For countries like Spain, where a spectrum of political ideologies coexists, understanding the sentiment behind political communications becomes crucial. Sentiment analysis, particularly on platforms like Twitter, serves as a powerful tool to decode public attitudes and the emotional undertones in political party communications (Cambria et al., 2013; Giachanou & Crestani, 2016). By leveraging sentiment analysis, researchers can quantify and interpret political sentiments, thereby offering insights into party strategies and public reactions. In Spain's unique political landscape, where new and traditional parties like Podemos, PSOE, PP, Ciudadanos, and Vox engage vigorously on social media, analyzing sentiment can reveal the underlying strategies each employs. Recent advancements in pre-trained models tailored for the Spanish language, such as BETO and RoBERTuito, offer refined accuracy in detecting nuanced sentiments within Spanish tweets (Pérez et al., 2021).


Attention Masks Help Adversarial Attacks to Bypass Safety Detectors

arXiv.org Artificial Intelligence

Despite recent research advancements in adversarial attack methods, current approaches against XAI monitors are still discoverable and slower. In this paper, we present an adaptive framework for attention mask generation to enable stealthy, explainable and efficient PGD image classification adversarial attack under XAI monitors. Specifically, we utilize mutation XAI mixture and multitask self-supervised X-UNet for attention mask generation to guide PGD attack. Experiments on MNIST (MLP), CIFAR-10 (AlexNet) have shown that our system can outperform benchmark PGD, Sparsefool and SOTA SINIFGSM in balancing among stealth, efficiency and explainability which is crucial for effectively fooling SOTA defense protected classifiers.