Goto

Collaborating Authors

 loophole


Starmer to extend online safety rules to AI chatbots after Grok scandal

The Guardian

The government said it would close a legal loophole in the Online Safety Act. The government said it would close a legal loophole in the Online Safety Act. Starmer to announce'crackdown on vile illegal content created by AI' after scandal involving Elon Musk's Grok tool Makers of AI chatbots that put children at risk will face massive fines or even see their services blocked in the UK under law changes to be announced by Keir Starmer on Monday. Emboldened by Elon Musk's X stopping its Grok AI tool from creating sexualised images of real people in the UK after public outrage last month, ministers are planning a "crackdown on vile illegal content created by AI". With more and more children using chatbots for everything from help with their homework to mental health support, the government said it would "move fast to shut a legal loophole and force all AI chatbot providers to abide by illegal content duties in the Online Safety Act or face the consequences of breaking the law".


Language Models Identify Ambiguities and Exploit Loopholes

Choi, Jio, Bansal, Mohit, Stengel-Eskin, Elias

arXiv.org Artificial Intelligence

Studying the responses of large language models (LLMs) to loopholes presents a two-fold opportunity. First, it affords us a lens through which to examine ambiguity and pragmatics in LLMs, since exploiting a loophole requires identifying ambiguity and performing sophisticated pragmatic reasoning. Second, loopholes pose an interesting and novel alignment problem where the model is presented with conflicting goals and can exploit ambiguities to its own advantage. To address these questions, we design scenarios where LLMs are given a goal and an ambiguous user instruction in conflict with the goal, with scenarios covering scalar implicature, structural ambiguities, and power dynamics. We then measure different models' abilities to exploit loopholes to satisfy their given goals as opposed to the goals of the user. We find that both closed-source and stronger open-source models can identify ambiguities and exploit their resulting loopholes, presenting a potential AI safety risk. Our analysis indicates that models which exploit loopholes explicitly identify and reason about both ambiguity and conflicting goals.


Specification Self-Correction: Mitigating In-Context Reward Hacking Through Test-Time Refinement

Gallego, Víctor

arXiv.org Artificial Intelligence

Language models (LMs) are susceptible to in-context reward hacking, where they exploit flaws in tainted or faulty written specifications or rubrics to achieve high scores without fulfilling the user's true intent. We introduce Specification Self-Correction (SSC), a novel, test-time framework that enables an LM to identify and correct flaws within its own guiding specification. SSC employs a multi-step inference process where the model first generates a response based on a potentially tainted specification, critiques its output, and then revises the specification itself to remove the exploitable loophole. A final, more robust response is then generated using this self-corrected specification. Across experiments spanning creative writing and agentic coding tasks with several LMs, we demonstrate that while models initially game tainted specifications in 50-70\% of cases, the SSC process reduces this vulnerability by over 90\%. This dynamic repair occurs at inference time, requires no weight modification, and leads to more robustly aligned model behavior. Code at https://github.com/vicgalle/specification-self-correction .


Can AI expose tax loopholes? Towards a new generation of legal policy assistants

Fratrič, Peter, Holzenberger, Nils, Amariles, David Restrepo

arXiv.org Artificial Intelligence

The legislative process is the backbone of a state built on solid institutions. Yet, due to the complexity of laws -- particularly tax law -- policies may lead to inequality and social tensions. In this study, we introduce a novel prototype system designed to address the issues of tax loopholes and tax avoidance. Our hybrid solution integrates a natural language interface with a domain-specific language tailored for planning. We demonstrate on a case study how tax loopholes and avoidance schemes can be exposed. We conclude that our prototype can help enhance social welfare by systematically identifying and addressing tax gaps stemming from loopholes.


Practical Principles for AI Cost and Compute Accounting

Casper, Stephen, Bailey, Luke, Schreier, Tim

arXiv.org Artificial Intelligence

Policymakers are increasingly using development cost and compute as proxies for AI model capabilities and risks. Recent laws have introduced regulatory requirements that are contingent on specific thresholds. However, technical ambiguities in how to perform this accounting could create loopholes that undermine regulatory effectiveness. This paper proposes seven principles for designing practical AI cost and compute accounting standards that (1) reduce opportunities for strategic gaming, (2) avoid disincentivizing responsible risk mitigation, and (3) enable consistent implementation across companies and jurisdictions.


EU accused of leaving 'devastating' copyright loophole in AI Act

The Guardian

"What I do not understand is that we are supporting big tech instead of protecting European creative ideas and content." The EU's AI Act, which came into force last year, was already in the works when ChatGPT, an AI chatbot that can generate essays, jokes and job applications, burst into public consciousness in late 2022, becoming the fastest-growing consumer application in history. ChatGPT was developed by OpenAI, which is also behind the AI image generator Dall-E. He would like legislation to fill that gap, but said it would take years, after the European Commission's decision last week to withdraw the proposed AI Liability Act. "It might be getting very difficult.


Defending Compute Thresholds Against Legal Loopholes

Pistillo, Matteo, Villalobos, Pablo

arXiv.org Artificial Intelligence

Existing legal frameworks on AI rely on training compute thresholds as a proxy to identify potentially-dangerous AI models and trigger increased regulatory attention. In the United States, Section 4.2(a) of Executive Order 14110 instructs the Secretary of Commerce to require extensive reporting from developers of AI models above a certain training compute threshold. In the European Union, Article 51 of the AI Act establishes a presumption that AI models above a certain compute threshold have high impact capabilities and hence pose systemic risk, thus subjecting their developers to several obligations including capability evaluations, reporting, and incident monitoring. In this paper, we examine some enhancement techniques that are capable of decreasing training compute usage while preserving, or even increasing, model capabilities. Since training compute thresholds rely on training compute as a metric and trigger for increased regulatory attention, these capability-enhancing and compute-saving techniques could constitute a legal loophole to existing training compute thresholds. In particular, we concentrate on four illustrative techniques (fine-tuning, model reuse, model expansion, and above compute-optimal inference compute) with the goal of furthering the conversation about their implications on training compute thresholds as a legal mechanism and advancing policy recommendations that could address the relevant legal loopholes.


TikTok's AI efforts reportedly exploit loopholes to use premium Nvidia chips

Engadget

The US has banned companies like Nvidia from selling their most advanced AI chips to China since 2022. But if loopholes exist, profit-hungry corporations will find and exploit them. The Information published a bombshell report on Thursday detailing how Oracle allows TikTok owner ByteDance to rent Nvidia's most advanced chips to train AI models on US soil. ByteDance, which many US lawmakers believe has direct ties to the Chinese government, is reportedly renting US-based servers containing Nvidia's coveted H100 chips from US cloud computing company Oracle to train AI models. The practice, which runs against the spirit of the US government's chip regulations, is technically allowed because Oracle is merely renting out the chips on American soil, not selling them to companies in China.


AI Procurement Checklists: Revisiting Implementation in the Age of AI Governance

Zick, Tom, Kortz, Mason, Eaves, David, Doshi-Velez, Finale

arXiv.org Artificial Intelligence

Public sector use of AI has been quietly on the rise for the past decade, but only recently have efforts to regulate it entered the cultural zeitgeist. While simple to articulate, promoting ethical and effective roll outs of AI systems in government is a notoriously elusive task. On the one hand there are hard-to-address pitfalls associated with AI-based tools, including concerns about bias towards marginalized communities, safety, and gameability. On the other, there is pressure not to make it too difficult to adopt AI, especially in the public sector which typically has fewer resources than the private sector$\unicode{x2014}$conserving scarce government resources is often the draw of using AI-based tools in the first place. These tensions create a real risk that procedures built to ensure marginalized groups are not hurt by government use of AI will, in practice, be performative and ineffective. To inform the latest wave of regulatory efforts in the United States, we look to jurisdictions with mature regulations around government AI use. We report on lessons learned by officials in Brazil, Singapore and Canada, who have collectively implemented risk categories, disclosure requirements and assessments into the way they procure AI tools. In particular, we investigate two implemented checklists: the Canadian Directive on Automated Decision-Making (CDADM) and the World Economic Forum's AI Procurement in a Box (WEF). We detail three key pitfalls around expertise, risk frameworks and transparency, that can decrease the efficacy of regulations aimed at government AI use and suggest avenues for improvement.


Robert F. Kennedy Jr.'s Microsoft-Powered Chatbot Just Disappeared

WIRED

Since Robert F. Kennedy Jr. first announced his longshot presidential bid, his campaign has leaned into a variety of unorthodox digital strategies. He's appeared on countless podcasts and has collabed with popular influencers to reach voters online. More recently, the Kennedy campaign has experimented with an AI chatbot that used an apparent loophole to get around OpenAI's restrictions on political use. On Sunday, after inquiries from WIRED, the chatbot disappeared. The loophole in question is an apparent result of the tight relationship between Microsoft and OpenAI.