Current evaluations of defenses against prompt attacks in large language model (LLM) applications often overlook two critical factors: the dynamic nature of adversarial behavior and the usability penalties imposed on legitimate users by restrictive defenses. We propose D-SEC (Dynamic Security Utility Threat Model), which explicitly separates attackers from legitimate users, models multi-step interactions, and rigorously expresses the security-utility in an optimizable form. We further address the shortcomings in existing evaluations by introducing Gandalf, a crowd-sourced, gamified red-teaming platform designed to generate realistic, adaptive attack datasets. Using Gandalf, we collect and release a dataset of 279k prompt attacks. Complemented by benign user data, our analysis reveals the interplay between security and utility, showing that defenses integrated in the LLM (e.g., system prompts) can degrade usability even without blocking requests. We demonstrate that restricted application domains, defense-in-depth, and adaptive defenses are effective strategies for building secure and useful LLM applications. Code is available at \href{https://github.com/lakeraai/dsec-gandalf}{\texttt{https://github.com/lakeraai/dsec-gandalf}}.

Large Language Models (LLMs) have emerged as the new recommendation engines, outperforming traditional methods in both capability and scope, particularly in code generation applications. Our research reveals a novel provider bias in LLMs, namely without explicit input prompts, these models show systematic preferences for services from specific providers in their recommendations (e.g., favoring Google Cloud over Microsoft Azure). This bias holds significant implications for market dynamics and societal equilibrium, potentially promoting digital monopolies. It may also deceive users and violate their expectations, leading to various consequences. This paper presents the first comprehensive empirical study of provider bias in LLM code generation. We develop a systematic methodology encompassing an automated pipeline for dataset generation, incorporating 6 distinct coding task categories and 30 real-world application scenarios. Our analysis encompasses over 600,000 LLM-generated responses across seven state-of-the-art models, utilizing approximately 500 million tokens (equivalent to \$5,000+ in computational costs). The study evaluates both the generated code snippets and their embedded service provider selections to quantify provider bias. Additionally, we conduct a comparative analysis of seven debiasing prompting techniques to assess their efficacy in mitigating these biases. Our findings demonstrate that LLMs exhibit significant provider preferences, predominantly favoring services from Google and Amazon, and can autonomously modify input code to incorporate their preferred providers without users' requests. Notably, we observe discrepancies between providers recommended in conversational contexts versus those implemented in generated code. The complete dataset and analysis results are available in our repository.

A stuttering economy, spiralling government borrowing costs, plummeting approval ratings: little wonder perhaps senior ministers, not least the chancellor, aren't wasting many smiles these days. Remember too Sir Keir Starmer and Rachel Reeves are the duo that best personify the Labour project of the 2020s; the party's revival and return, grounded in being trusted on the economy. And yet the markets are collectively passing a verdict on Starmer and Reeves's economic plan right now and it isn't exactly a ringing endorsement – and wobbly markets can prompt political wobbles. These shouldn't be overstated, but neither should they be ignored. What we're witnessing is the brutal slog of government playing out; an unforgiving backdrop of economic flatlining, which critics say ministers have made worse.

The FBI recently confirmed a Canadian plane offering assistance during the California wildfires was damaged in a collision with a privately-owned drone. Police arrested three people following two drone incidents as authorities report numerous encounters with aerial operations, potentially hampering lifesaving measures as wildfires rage throughout Southern California. As of Monday afternoon, charges had not been released. Two arrests stem from one drone incident, according to Los Angeles County Sheriff Robert Luna. "If you do not have business in the evacuation areas, do not go there," Luna said in a press conference on Monday.

Machine learning based traffic forecasting models leverage sophisticated spatiotemporal auto-correlations to provide accurate predictions of city-wide traffic states. However, existing methods assume a reliable and unbiased forecasting environment, which is not always available in the wild. In this work, we investigate the vulnerability of spatiotemporal traffic forecasting models and propose a practical adversarial spatiotemporal attack framework. Specifically, instead of simultaneously attacking all geo-distributed data sources, an iterative gradient guided node saliency method is proposed to identify the time-dependent set of victim nodes. Furthermore, we devise a spatiotemporal gradient descent based scheme to generate real-valued adversarial traffic states under a perturbation constraint.Meanwhile, we theoretically demonstrate the worst performance bound of adversarial traffic forecasting attacks.

Advances in the development of adversarial attacks have been fundamental to the progress of adversarial defense research. Efficient and effective attacks are crucial for reliable evaluation of defenses, and also for developing robust models. Adversarial attacks are often generated by maximizing standard losses such as the cross-entropy loss or maximum-margin loss within a constraint set using Projected Gradient Descent (PGD). In this work, we introduce a relaxation term to the standard loss, that finds more suitable gradient-directions, increases attack efficacy and leads to more efficient adversarial training. We propose Guided Adversarial Margin Attack (GAMA), which utilizes function mapping of the clean image to guide the generation of adversaries, thereby resulting in stronger attacks.

Personal health data is by its nature highly sensitive and its vulnerability in a digital environment has already been underlined by recent ransomware attacks that have affected NHS trusts. Andrew Duncan, the director of foundational AI at the UK's Alan Turing Institute, says even anonymised health data can be manipulated to identify a patient through a process known as "re-identification" whereby "de-identified" data can be matched to other available information to identify someone. "Once you start to narrow things down you can start to re-identify people easily," he says. Duncan adds that AI models can be trained in a way that prevents re-identification, although "the caveat is that all of this has to be done very carefully". MedConfidential, which campaigns for confidentiality in healthcare, also wants clarity on whether a health dataset will respect patients who have signed an opt-out that prevents their data being used for research and planning in England.

The outgoing administration of United States President Joe Biden is proposing a new framework for the export of advanced computer chips used to develop artificial intelligence, an attempt to balance national security concerns about the technology with the economic interests of producers and other countries. But the framework proposed Monday also raised concerns of chip industry executives who said the rules would limit access to existing chips used for video games and restrict in 120 countries the chips used for data centres and AI products. Mexico, Portugal, Israel and Switzerland are among the nations that could have limited access. Commerce Secretary Gina Raimondo said on a call with reporters previewing the framework that it's "critical" to preserve America's leadership in AI and the development of AI-related computer chips. Fast-evolving AI technology enables computers to produce novels, make scientific research breakthroughs, automate driving and foster a range of other transformations that could reshape economies and warfare.

The Biden Administration's move on Jan. 13 to curb exports on the advanced computer chips used to power artificial intelligence (AI) arrived in the wake of two major events over the Christmas holidays that rattled the world of AI. First, OpenAI released its latest model, o3, which achieved an 88% on a set of difficult reasoning tests on which no AI system had previously scored above 5%. "All intuition about AI capabilities will need to get updated" in light of the results, said Francois Chollet, a former AI researcher at Google and a prominent skeptic of the argument that "artificial general intelligence" (AGI) would be achieved any time soon. Second, the Chinese company DeepSeek released an open-source AI model that outperformed any American open-source language model, including Meta's Llama series. The achievement surprised many AI researchers and U.S. officials, who had believed China lagged behind in terms of AI capabilities.

During his speech, Sir Keir emphasised that his government would take a more aggressive approach to the development of AI than the European Union has. Major tech companies have previously criticised the EU's approach claiming that it hinders growth and hampers innovation. Under the AI Act introduced by the EU last August, systems considered "high-risk", in critical infrastructure, education, healthcare, law enforcement, border management or elections, will have to comply with strict requirements set out by lawmakers. Sir Keir says the UK would "go our own way on this", and would regulate in a way that was "pro-growth and pro-innovation". And while the prime minister said he backed the AI Safety Institute set up by the last government, he suggested the Conservatives may have adopted an overbearing approach to AI safety.