Goto

Collaborating Authors

 guardrail


Trump dismisses calls for AI slowdown from leading tech CEOs

Al Jazeera

US President Donald Trump has downplayed the need for his administration to place checks on artificial intelligence development, saying he is worried about ceding the US's lead to China. Speaking to reporters during his trip to Ireland on Sunday, Trump also said he acknowledged the need for some regulation but did not provide further details on potential measures. "We can put guardrails, we can do this and that, but I think you have a lot of negative forces that are bringing it up that shouldn't be bringing it up, and they're bringing up things that won't happen," Trump told reporters after watching the Irish Open at his resort in Doonbeg. He said that with the US leading the development of AI over China, "frankly, I want to keep it that way because whoever wins AI wins." Trump's comments came a day after Anthropic CEO Dario Amodei published a blog post saying the AI industry must slow the pace of improving the technology's capabilities to allow time for guardrails.


I Let an AI Agent Hack All My Gadgets--and I'd Do It Again

WIRED

I Let an AI Agent Hack All My Gadgets--and I'd Do It Again After I removed the safety guardrails from a powerful open-source model, it found vulnerabilities in my household devices and hacked into a PC. But it also told me how to make everything a lot more secure. As the author of a newsletter about artificial intelligence, I consider it my duty to experience the bleeding edge of this technology firsthand. This week, that meant embracing some agentic mayhem. You're probably aware that frontier AI models have attained advanced cybersecurity capabilities in recent months.


OpenAI chief scientist warns no one is prepared for consequences of AI

BBC News

OpenAI's chief scientist Jakub Pachocki has called for extreme caution over AI's runaway progress and warned more intervention may be needed to ensure humans remain in control of the future. I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence, he wrote in a blog post entitled An Alien Mind, external . The post comes only a few days after the ChatGPT-maker released its latest model GPT-6 Astra, which it called its most powerful product ever. OpenAI and other AI firms like Anthropic shared reports of their AI agents acting autonomously and carrying out real-world cyber-attacks on other companies. In July, OpenAI called an incident in which its AI agents - AI systems which can operate alone after human instruction - hacked the tech platform Hugging Face unprecedented.


One of science fiction's greatest writers warned us about a AI. Does he also hold the remedy? Alan Finkel

The Guardian

'Implementing these laws on AI would be difficult, but the stakes are existential and therefore the effort is worthwhile,' writes Alan Finkel Photograph: Alexey Kotelnikov/Alamy'Implementing these laws on AI would be difficult, but the stakes are existential and therefore the effort is worthwhile,' writes Alan Finkel Photograph: Alexey Kotelnikov/Alamy One of science fiction's greatest writers warned us about AI. Could his ideas help us avoid a dystopian future? What might a modern day equivalent of Isaac Asimov's laws of robotics look like? Fri 7 Aug 2026 00.00 EDTLast modified on Fri 7 Aug 2026 11.29 EDT Tesla and SpaceX founder Elon Musk predicted in July that legions of AI-powered robots would dominate the physical world and that AI might not take orders from people any more. He also offered an alternative vision in which there would be agreement for a collective objective to make AI benign by imbuing it with a love of the truth and a desire for humanity to prosper, and that governments might have to enforce this objective.


OpenAI's models autonomously hacked a tech startup. It signals a seismic shift in cybersecurity

AIHub

An autonomous agent powered by OpenAI's advanced artificial intelligence (AI) models went rogue during a security test and hacked multi-billion dollar tech startup, Hugging Face, last week. The agent didn't just exploit vulnerabilities in Hugging Face's systems to achieve what it perceived as a strategic gain. It also exploited vulnerabilities within OpenAI's infrastructure. Of course, hacks are very common cyber threats that organisations face frequently. But this incident is different, because the AI agent acted without any human input.


Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency

Neural Information Processing Systems

Despite their superior performance on a wide range of domains, large language models (LLMs) remain vulnerable to misuse for generating harmful content, a risk that has been further amplified by various jailbreak attacks. Existing jailbreak attacks mainly follow sequential logic, where LLMs understand and answer each given task one by one. However, concurrency, a natural extension of the sequential scenario, has been largely overlooked. In this work, we first propose a wordlevel method to enable task concurrency in LLMs, where adjacent words encode divergent intents. Although LLMs maintain strong utility in answering concurrent tasks, which is demonstrated by our evaluations on mathematical and general question-answering benchmarks, we notably observe that combining a harmful task with a benign one significantly reduces the probability of it being filtered by the guardrail, showing the potential risks associated with concurrency in LLMs. Based on these findings, we introduce JAIL-CON, an iterative attack framework that JAILbreaks LLMs via task CONcurrency. Experiments on widely-used LLMs demonstrate the strong jailbreak capabilities of JAIL-CON compared to existing attacks. Furthermore, when the guardrail is applied as a defense, compared to the sequential answers generated by previous attacks, the concurrent answers in our JAIL-CONexhibit greater stealthiness and are less detectable by the guardrail, highlighting the unique feature of task concurrency in jailbreaking LLMs.1 Disclaimer: This paper contains unsafe information.


From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoring

Neural Information Processing Systems

Warning: this paper may contain potentially generated harmful content. Though safety alignment has been applied to most large language models (LLMs), LLM service providers generally deploy a subsequent moderation as the external safety guardrail in real-world products. Existing moderators mainly practice a conventional full detection, which determines the harmfulness based on the complete LLM output, causing high service latency. Recent works pay more attention to partial detection where moderators oversee the generation midway and early stop the output if harmfulness is detected, but they directly apply moderators trained with the full detection paradigm to incomplete outputs, introducing a training-inference gap that lowers the performance. In this paper, we explore how to form a data-andmodel solution that natively supports partial detection. For the data, we construct FineHarm, a dataset consisting of 29K prompt-response pairs with fine-grained token-level annotations to provide reasonable supervision for token-level training. Then, we propose the Streaming Content Monitor (SCM), which is trained with dual supervision of response-and token-level labels and can follow the output stream of LLM to make a timely judgment of harmfulness. Experiments show that SCM gains 0.95+ in macro F1 score that is comparable to full detection, by only seeing the first 18% of tokens in responses on average. Moreover, the SCM can serve as a pseudo-harmfulness annotator for improving safety alignment and lead to a higher harmlessness score than DPO.


Anthropic Is Still at Odds With the White House Over Claude Fable 5

WIRED

Anthropic leaders flew to Washington, DC, to meet with White House officials on Monday. Trump administration officials concluded talks with Anthropic on Monday without lifting export controls that were imposed last week on the company's most advanced AI models in response to jailbreaking concerns, according to three people briefed on the matter. The administration continues to believe that there are ways to disable some of the guardrails on Anthropic's Claude Fable 5, effectively allowing users to access the more powerful cybersecurity capabilities of the company's Mythos model, the people said. Anthropic has said for days that the administration's concerns are overblown, a position it reiterated in working group meetings held at the Commerce Department with government researchers from Center for AI Standards and Innovation (CAISI) and the Office of the National Cyber Director Sean Cairncross, one of the people said. The meetings were also attended by Commerce secretary Howard Lutnick, who dialed in by conference call from the G7 summit in Evian, France.


Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency

Neural Information Processing Systems

Despite their superior performance on a wide range of domains, large language models (LLMs) remain vulnerable to misuse for generating harmful content, a risk that has been further amplified by various jailbreak attacks. Existing jailbreak attacks mainly follow sequential logic, where LLMs understand and answer each given task one by one. However, concurrency, a natural extension of the sequential scenario, has been largely overlooked. In this work, we first propose a word-level method to enable task concurrency in LLMs, where adjacent words encode divergent intents. Although LLMs maintain strong utility in answering concurrent tasks, which is demonstrated by our evaluations on mathematical and general question-answering benchmarks, we notably observe that combining a harmful task with a benign one significantly reduces the probability of it being filtered by the guardrail, showing the potential risks associated with concurrency in LLMs. Based on these findings, we introduce $\texttt{JAIL-CON}$, an iterative attack framework that $\underline{\text{JAIL}}$breaks LLMs via task $\underline{\text{CON}}$currency. Experiments on widely-used LLMs demonstrate the strong jailbreak capabilities of $\texttt{JAIL-CON}$ compared to existing attacks. Furthermore, when the guardrail is applied as a defense, compared to the sequential answers generated by previous attacks, the concurrent answers in our $\texttt{JAIL-CON}$ exhibit greater stealthiness and are less detectable by the guardrail, highlighting the unique feature of task concurrency in jailbreaking LLMs.


The Meta hack shows there's more to AI security than Mythos

MIT Technology Review

On June 5, reported that attackers had been using Meta's AI customer support agent to steal Instagram accounts. Their approach was simple: They asked the agent to link the accounts to email addresses that they controlled, and the agent complied. One attacker broke into the dormant Obama White House account and made pro-Iran posts; others took over accounts with valuable, single-word handles, possibly in order to sell them. AI cybersecurity concerns are nothing new. Since Anthropic announced in April that its Mythos model was too good at hacking to be released to the general public, commentators, researchers, and federal officials alike have fixated on the idea that superpowered AI systems could lay waste to our computer infrastructure. That's not quite what this Instagram hack was: There, AI was the target rather than the attacker, and the method was far simpler than anything Mythos would cook up. But as companies offload more work to AI, these comparatively unsophisticated attacks could wreak their own havoc. "As AI becomes more and more widely used--especially when AI is more and more widely used to automate our work flows, like account recovery--I think attackers are going to be more and more motivated to attack AI itself," says Neil Gong, a professor of electrical and computer engineering at Duke University.