Goto

Collaborating Authors

 Government


Sentiment Simulation using Generative AI Agents

arXiv.org Artificial Intelligence

Traditional sentiment analysis relies on surface-level linguistic patterns and retrospective data, limiting its ability to capture the psychological and contextual drivers of human sentiment. These limitations constrain its effectiveness in applications that require predictive insight, such as policy testing, narrative framing, and behavioral forecasting. We present a robust framework for sentiment simulation using generative AI agents embedded with psychologically rich profiles. Agents are instantiated from a nationally representative survey of 2,485 Filipino respondents, combining sociodemographic information with validated constructs of personality traits, values, beliefs, and socio-political attitudes. The framework includes three stages: (1) agent embodiment via categorical or contextualized encodings, (2) exposure to real-world political and economic scenarios, and (3) generation of sentiment ratings accompanied by explanatory rationales. Using Quadratic Weighted Accuracy (QWA), we evaluated alignment between agent-generated and human responses. Contextualized encoding achieved 92% alignment in replicating original survey responses. In sentiment simulation tasks, agents reached 81%--86% accuracy against ground truth sentiment, with contextualized profile encodings significantly outperforming categorical (p < 0.0001, Cohen's d = 0.70). Simulation results remained consistent across repeated trials (+/-0.2--0.5% SD) and resilient to variation in scenario framing (p = 0.9676, Cohen's d = 0.02). Our findings establish a scalable framework for sentiment modeling through psychographically grounded AI agents. This work signals a paradigm shift in sentiment analysis from retrospective classification to prospective and dynamic simulation grounded in psychology of sentiment formation.


Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate

arXiv.org Artificial Intelligence

Large language models (LLMs) provide detailed and impressive responses to queries in English. However, are they really consistent at responding to the same query in other languages? The popular way of evaluating for multilingual performance of LLMs requires expensive-to-collect annotated datasets. Further, evaluating for tasks like open-ended generation, where multiple correct answers may exist, is nontrivial. Instead, we propose to evaluate the predictability of model response across different languages. In this work, we propose a framework to evaluate LLM's cross-lingual consistency based on a simple Translate then Evaluate strategy. We instantiate this evaluation framework along two dimensions of consistency: information and empathy. Our results reveal pronounced inconsistencies in popular LLM responses across thirty languages, with severe performance deficits in certain language families and scripts, underscoring critical weaknesses in their multilingual capabilities. These findings necessitate cross-lingual evaluations that are consistent along multiple dimensions. We invite practitioners to use our framework for future multilingual LLM benchmarking.


A Joint Reconstruction-Triplet Loss Autoencoder Approach Towards Unseen Attack Detection in IoV Networks

arXiv.org Artificial Intelligence

Internet of Vehicles (IoV) systems, while offering significant advancements in transportation efficiency and safety, introduce substantial security vulnerabilities due to their highly interconnected nature. These dynamic systems produce massive amounts of data between vehicles, infrastructure, and cloud services and present a highly distributed framework with a wide attack surface. In considering network-centered attacks on IoV systems, attacks such as Denial-of-Service (DoS) can prohibit the communication of essential physical traffic safety information between system elements, illustrating that the security concerns for these systems go beyond the traditional confidentiality, integrity, and availability concerns of enterprise systems. Given the complexity and volume of data generated by IoV systems, traditional security mechanisms are often inadequate for accurately detecting sophisticated and evolving cyberattacks. Here, we present an unsupervised autoencoder method trained entirely on benign network data for the purpose of unseen attack detection in IoV networks. We leverage a weighted combination of reconstruction and triplet margin loss to guide the autoencoder training and develop a diverse representation of the benign training set. We conduct extensive experiments on recent network intrusion datasets from two different application domains, industrial IoT and home IoT, that represent the modern IoV task. We show that our method performs robustly for all unseen attack types, with roughly 99% accuracy on benign data and between 97% and 100% performance on anomaly data. We extend these results to show that our model is adaptable through the use of transfer learning, achieving similarly high results while leveraging domain features from one domain to another.


Preventing Adversarial AI Attacks Against Autonomous Situational Awareness: A Maritime Case Study

arXiv.org Artificial Intelligence

Adversarial artificial intelligence (AI) attacks pose a significant threat to autonomous transportation, such as maritime vessels, that rely on AI components. Malicious actors can exploit these systems to deceive and manipulate AI-driven operations. This paper addresses three critical research challenges associated with adversarial AI: the limited scope of traditional defences, inadequate security metrics, and the need to build resilience beyond model-level defences. To address these challenges, we propose building defences utilising multiple inputs and data fusion to create defensive components and an AI security metric as a novel approach toward developing more secure AI systems. We name this approach the Data Fusion Cyber Resilience (DFCR) method, and we evaluate it through real-world demonstrations and comprehensive quantitative analyses, comparing a system built with the DFCR method against single-input models and models utilising existing state-of-the-art defences. The findings show that the DFCR approach significantly enhances resilience against adversarial machine learning attacks in maritime autonomous system operations, achieving up to a 35\% reduction in loss for successful multi-pronged perturbation attacks, up to a 100\% reduction in loss for successful adversarial patch attacks and up to 100\% reduction in loss for successful spoofing attacks when using these more resilient systems. We demonstrate how DFCR and DFCR confidence scores can reduce adversarial AI contact confidence and improve decision-making by the system, even when typical adversarial defences have been compromised. Ultimately, this work contributes to the development of more secure and resilient AI-driven systems against adversarial attacks.


Public Discourse Sandbox: Facilitating Human and AI Digital Communication Research

arXiv.org Artificial Intelligence

Social media serves as a primary communication and information dissemination platform for major global events, entertainment, and niche or topically focused community discussions. Therefore, it represents a valuable resource for researchers who aim to understand numerous questions. However, obtaining data can be difficult, expensive, and often unreliable due to the presence of bots, fake accounts, and manipulated content. Additionally, there are ethical concerns if researchers decide to conduct an online experiment without explicitly notifying social media users about their intent. There is a need for more controlled and scalable mechanisms to evaluate the impacts of digital discussion interventions on audiences. We introduce the Public Discourse Sandbox (PDS), which serves as a digital discourse research platform for human-AI as well as AI-AI discourse research, testing, and training. PDS provides a safe and secure space for research experiments that are not viable on public, commercial social media platforms. Its main purpose is to enable the understanding of AI behaviors and the impacts of customized AI participants via techniques such as prompt engineering, retrieval-augmented generation (RAG), and fine-tuning. We provide a hosted live version of the sandbox to support researchers as well as the open-sourced code on GitHub for community collaboration and contribution.


More than 1bn earmarked for battlefield tech

BBC News

Announcing the results of the review, the MoD said a new Digital Targeting Web would better connect soldiers on the ground with key information provided by satellites, aircraft and drones helping them target enemy threats faster. Defence Secretary John Healey said the technology announced in the review - which will harness Artificial Intelligence (AI) and software - also highlights lessons being learnt from the war in Ukraine. Ukraine is already using AI and software to speed up the process of identifying, and then hitting, Russian military targets. The review had been commissioned by the newly formed Labour government shortly after last year's election with Healey describing it as the "first of its kind". The government said the findings would be published in the first half of 2025, but did not give an exact date.


Nvidia beats Wall Street expectations even as Trump tamps down China sales

The Guardian

Nvidia beat Wall Street expectations in its quarterly earnings report on Wednesday, marking another in a string of financial wins for the computer hardware giant. It reported 44.1bn in revenue in the quarter ending in April, up 69% from the previous year. The company exceeded investors' predictions of 43.3bn in revenue. Adjusted earnings per share came in at 0.81, under investor expectations of an adjusted earnings per share of 88 cents. The company also reported 39.1bn in data center revenue, up 73% from the year prior.


Why Anthropic's New AI Model Sometimes Tries to 'Snitch'

WIRED

Anthropic's alignment team was doing routine safety testing in the weeks leading up to the release of its latest AI models when researchers discovered something unsettling: When one of the models detected that it was being used for "egregiously immoral" purposes, it would attempt to "use command-line tools to contact the press, contact regulators, try to lock you out of the relevant systems, or all of the above," researcher Sam Bowman wrote in a post on X last Thursday. Bowman deleted the post shortly after he shared it, but the narrative about Claude's whistleblower tendencies had already escaped containment. "Claude is a snitch," became a common refrain in some tech circles on social media. At least one publication framed it as an intentional product feature rather than what it was--an emergent behavior. "It was a hectic 12 hours or so while the Twitter wave was cresting," Bowman tells WIRED.


A new law in this state bans automated insurance claim denials

FOX News

'Ask Dr. Drew' host Dr. Drew Pinsky breaks down key takeaways from the MAHA Commission's chronic disease report on'The Ingraham Angle.' As some health insurance companies have come under fire for allegedly using computer systems to shoot down claims, an Arizona law will soon make the practice illegal in the Grand Canyon State. Republican Arizona House Majority Whip Rep. Julie Willoughby sponsored the legislation, and it was recently signed into law by Democratic Gov. Katie Hobbs. House Bill 2175 requires a physician licensed in the state to conduct an "individual review" and use "independent medical judgment" to determine whether the claim should actually be denied. It also required a similar review of "a direct denial of a prior authorization of a service" that a provider asked for and "involves medical necessity."


Elon's Twitter Purchase Turned Out to Be a Great Investment--but Not for the Reasons You Think

Slate

Sign up for the Slatest to get the most insightful analysis, criticism, and advice out there, delivered to your inbox daily. Through a stroke of good fortune, Elon Musk's otherwise disastrous purchase of Twitter has turned into one of the great business acquisitions of all time. Buying control of a president was a start. What if the deal bought him something even more valuable? Musk's purchase of Twitter, which closed in the fall of 2022, has undergone an odyssey.