Goto

Collaborating Authors

 Government


Order Independence With Finetuning

arXiv.org Artificial Intelligence

Large language models (LLMs) demonstrate remarkable performance on many NLP tasks, yet often exhibit order dependence: simply reordering semantically identical tokens (e.g., answer choices in multiple-choice questions) can lead to inconsistent predictions. Recent work proposes Set-Based Prompting (SBP) as a way to remove order information from designated token subsets, thereby mitigating positional biases. However, applying SBP on base models induces an out-of-distribution input format, which can degrade in-distribution performance. We introduce a fine-tuning strategy that integrates SBP into the training process, "pulling" these set-formatted prompts closer to the model's training manifold. We show that SBP can be incorporated into a model via fine-tuning. Our experiments on in-distribution (MMLU) and out-of-distribution (CSQA, ARC Challenge) multiple-choice tasks show that SBP fine-tuning significantly improves accuracy and robustness to answer-order permutations, all while preserving broader language modeling capabilities. We discuss the broader implications of order-invariant modeling and outline future directions for building fairer, more consistent LLMs.


Mapping Geopolitical Bias in 11 Large Language Models: A Bilingual, Dual-Framing Analysis of U.S.-China Tensions

arXiv.org Artificial Intelligence

This study systematically analyzes geopolitical bias across 11 prominent Large Language Models (LLMs) by examining their responses to seven critical topics in U.S.-China relations. Utilizing a bilingual (English and Chinese) and dual-framing (affirmative and reverse) methodology, we generated 19,712 prompts designed to detect ideological leanings in model outputs. Responses were quantitatively assessed on a normalized scale from -2 (strongly Pro-China) to +2 (strongly Pro-U.S.) and categorized according to stance, neutrality, and refusal rates. The findings demonstrate significant and consistent ideological alignments correlated with the LLMs' geographic origins; U.S.-based models predominantly favored Pro-U.S. stances, while Chinese-origin models exhibited pronounced Pro-China biases. Notably, language and prompt framing substantially influenced model responses, with several LLMs exhibiting stance reversals based on prompt polarity or linguistic context. Additionally, we introduced comprehensive metrics to evaluate response consistency across languages and framing conditions, identifying variability and vulnerabilities in model behaviors. These results offer practical insights that can guide organizations and individuals in selecting LLMs best aligned with their operational priorities and geopolitical considerations, underscoring the importance of careful model evaluation in politically sensitive applications. Furthermore, the research highlights specific prompt structures and linguistic variations that can strategically trigger distinct responses from models, revealing methods for effectively navigating and influencing LLM outputs.


Autonomous Learning with High-Dimensional Computing Architecture Similar to von Neumann's

arXiv.org Artificial Intelligence

We model human and animal learning by computing with high-dimensional vectors (H = 10,000 for example). The architecture resembles traditional (von Neumann) computing with numbers, but the instructions refer to vectors and operate on them in superposition. The architecture includes a high-capacity memory for vectors, analogue of the random-access memory (RAM) for numbers. The model's ability to learn from data reminds us of deep learning, but with an architecture closer to biology. The architecture agrees with an idea from psychology that human memory and learning involve a short-term working memory and a long-term data store. Neuroscience provides us with a model of the long-term memory, namely, the cortex of the cerebellum. With roots in psychology, biology, and traditional computing, a theory of computing with vectors can help us understand how brains compute. Application to learning by robots seems inevitable, but there is likely to be more, including language. Ultimately we want to compute with no more material and energy than used by brains. To that end, we need a mathematical theory that agrees with psychology and biology, and is suitable for nanotechnology. We also need to exercise the theory in large-scale experiments. Computing with vectors is described here in terms familiar to us from traditional computing with numbers.


Addressing Model Overcomplexity in Drug-Drug Interaction Prediction With Molecular Fingerprints

arXiv.org Artificial Intelligence

Accurately predicting drug-drug interactions (DDIs) is crucial for pharmaceutical research and clinical safety. Recent deep learning models often suffer from high computational costs and limited generalization across datasets. In this study, we investigate a simpler yet effective approach using molecular representations such as Morgan fingerprints (MFPS), graph-based embeddings from graph convolutional networks (GCNs), and transformer-derived embeddings from MoLFormer integrated into a straightforward neural network. We benchmark our implementation on DrugBank DDI splits and a drug-drug affinity (DDA) dataset from the Food and Drug Administration. MFPS along with MoLFormer and GCN representations achieve competitive performance across tasks, even in the more challenging leak-proof split, highlighting the sufficiency of simple molecular representations. Moreover, we are able to identify key molecular motifs and structural patterns relevant to drug interactions via gradient-based analyses using the representations under study. Despite these results, dataset limitations such as insufficient chemical diversity, limited dataset size, and inconsistent labeling impact robust evaluation and challenge the need for more complex approaches. Our work provides a meaningful baseline and emphasizes the need for better dataset curation and progressive complexity scaling.


Towards Trustworthy GUI Agents: A Survey

arXiv.org Artificial Intelligence

GUI agents, powered by large foundation models, can interact with digital interfaces, enabling various applications in web automation, mobile navigation, and software testing. However, their increasing autonomy has raised critical concerns about their security, privacy, and safety. This survey examines the trustworthiness of GUI agents in five critical dimensions: security vulnerabilities, reliability in dynamic environments, transparency and explainability, ethical considerations, and evaluation methodologies. We also identify major challenges such as vulnerability to adversarial attacks, cascading failure modes in sequential decision-making, and a lack of realistic evaluation benchmarks. These issues not only hinder real-world deployment but also call for comprehensive mitigation strategies beyond task success. As GUI agents become more widespread, establishing robust safety standards and responsible development practices is essential. This survey provides a foundation for advancing trustworthy GUI agents through systematic understanding and future research.


What Makes an Evaluation Useful? Common Pitfalls and Best Practices

arXiv.org Artificial Intelligence

Following the rapid increase in Artificial Intelligence (AI) capabilities in recent years, the AI community has voiced concerns regarding possible safety risks. To support decision-making on the safe use and development of AI systems, there is a growing need for high-quality evaluations of dangerous model capabilities. While several attempts to provide such evaluations have been made, a clear definition of what constitutes a "good evaluation" has yet to be agreed upon. In this practitioners' perspective paper, we present a set of best practices for safety evaluations, drawing on prior work in model evaluation and illustrated through cybersecurity examples. We first discuss the steps of the initial thought process, which connects threat modeling to evaluation design. Then, we provide the characteristics and parameters that make an evaluation useful. Finally, we address additional considerations as we move from building specific evaluations to building a full and comprehensive evaluation suite.


Four in Ukraine killed in drone strike as Russia claims advances on ground

Al Jazeera

A Russian drone attack has killed at least four people and wounded 21 in the eastern Ukrainian city of Dnipro, damaging high-rise buildings and triggering fires in a hotel and homes, the regional governor said, as Moscow claims to have made gains on the ground elsewhere. Late Friday, Russia sent "more than two dozen drones" to Dnipro, the governor of the surrounding Dnipropetrovsk region, Sergiy Lysak, wrote on his official Telegram account on Saturday. "The massive attack caused large-scale destruction and fires. A hotel and restaurant complex, 11 private houses, garages, and a service station were on fire," he said, adding that high-rises and cars were also damaged. Pictures and videos posted online showed flames and large plumes of smoke wafting skyward.


GenAI, the future of fraud and why you may be an easy target

FOX News

Don't let fraudsters create a false sense of urgency. If you receive a communication claiming to be from a financial institution, call that institution directly using the official number from its website.


Move fast, kill things: the tech startups trying to reinvent defence with Silicon Valley values

The Guardian

Visit tech startup Skydio's headquarters on the San Francisco peninsula in California and you're likely to find flying robots buzzing on the roof overhead. Docking stations with motorised covers open to allow small drones that resemble the TIE fighters from Star Wars films to take off; when each drone lands back again, they close. The drones can fly completely autonomously and without GPS, taking in data from onboard cameras and using AI to execute programmed missions and avoid obstacles. Skydio, with more than 740m in venture capital funding and a valuation of about 2.5bn, makes drones for the military along with civilian organisations such as police forces and utility companies. The company moved away from the consumer market in 2020 and is now the largest US drone maker.


Fox News AI Newsletter: North Korea's suicide drone test

FOX News

North Korean leader Kim Jong Un supervises the test of suicide drones with artificial intelligence technology, according to local media, at an unknown location, in this photo released by North Korea's official Korean Central News Agency on March 27, 2025. KIM POWER PLAY: North Korean dictator Kim Jong Un oversaw tests of newly developed AI-powered suicide drones and called for their increased production, North Korean state media said Thursday. A photo taken on October 4, 2023 in Manta, near Turin, shows a smartphone and a laptop displaying the logos of the artificial intelligence OpenAI research company and ChatGPT chatbot. SUZANNE'S TWIN: Suzanne Somers passed away two years ago, but her memory lives on, not only through her Hollywood career and businesses, but artificial intelligence too. Her widower, Alan Hamel, worked with an AI company called Hollo to create a "twin" of his late wife.