Goto

Collaborating Authors

 oversight


India bypasses own rules to buy Israeli phone-hacking tech

Al Jazeera

An investigation by the digital-rights group Internet Freedom Foundation (IFF) found Indian Home Minister Amit Shah, a close confidante of Prime Minister Narendra Modi, signed off on exemptions from "make in India" procurement rules, clearing the way for government forensic institutions to buy at least 422 items of foreign technology, including products made by the Israeli firm Cellebrite. Modi's federal government and state governments ruled by his Bharatiya Janata Party are not the only ones who bought this Israeli technology. States governed by the opposition Indian National Congress party, including Himachal Pradesh, Keralam and Karnataka, also purchased similar phone-extraction and cloning tools. A review of documents revealed that India's expansive surveillance capabilities extend far beyond just cloning phones and extracting data without passwords to accessing cloud-based data from social media platforms - such as Instagram, X and Tinder - as well as data from connected devices such as Amazon's Alexa and Google Home. Experts argued that India's current laws offer no guardrails against overreaching surveillance and the data extractions happen in a legal grey zone.


Pairing industrial AI autonomy and human control

MIT Technology Review

Content from MIT Technology Review Insights This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review's editorial staff. Living digital twins are helping industrial plants run more like living systems, and setting the stage for industries that work in sync. Functionally, though, a plant or factory is more like a living thing: It has a pulse, even a metabolism. It goes through wear and tear as it ages, and signs of deteriorating health often show up as barely perceptible irregularities.


Health Care Workers Are Tired of Cleaning Up Palantir's Mess

WIRED

Health Care Workers Are Tired of Cleaning Up Palantir's Mess A hospital giant and radiology network turned to Palantir to streamline scheduling, but nurses and other staff say the new software is causing errors, burnout, and frustration. Over the past four months, Amber Retzloff, a critical care nurse in Florida, requested to work 50 specific 12-hour shifts. But over half the time, she says her hospital's Palantir -powered scheduling software assigned her to different shifts, often putting her on back-to-back-to-back days and leaving her mentally drained. Retzloff doesn't expect to always get the slots she wants, but by her count the tool does a far worse job honoring her preferences than managers did when they assigned shifts by hand. Retzloff's experience mirrors that of other nurses across HCA Healthcare, the largest hospital chain in the US.


Meta-Led Anti-Terrorism Group Faces Mass Resignation of Expert Advisers

WIRED

Meta and other tech giants are pushing through structural changes at the Global Internet Forum to Counter Terrorism that outside researchers allege will weaken oversight of an already embattled consortium. Six of the nine independent experts on the advisory board of the Global Internet Forum to Counter Terrorism --a consortium run by several of the biggest US tech companies--resigned on Monday, according to a letter seen by WIRED and interviews with four of the people. The tensions between the independent advisory committee and the GIFCT date back to an email the counterterrorism and free speech experts received in July from Meta's Nell McCarthy, a vice president overseeing content policy. For years, the group had advised the GIFCT on how to prevent platforms from becoming havens for the radical organizations and individuals blamed for some of the world's worst mass violence. But McCarthy wrote that while the consortium welcomed the experts' insights on violent trends, it no longer desired their scrutiny on the effectiveness of Big Tech's efforts to curtail violence.


OpenAI, Anthropic CEOs call for global AI regulation at UN

Al Jazeera

The heads of several major AI firms told the United Nations Security Council (UNSC) their industry urgently needed global oversight to avoid dangers that could threaten the whole world. "If managed poorly, I even believe AI could be a risk to humanity as a whole," Dario Amodei, the chief executive officer of Anthropic, told members of the body on Wednesday. The meeting, which coincides with the UN General Assembly (UNGA) gathering in New York City, was convened by France and comes at a time when experts are increasingly warning that the rapid development of AI needs more human oversight to ensure it does not slip out of control and cause a global catastrophe. Altman and Amodei called on world leaders to take action. "If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone," Altman told members.


The Debate Moment That Captures Democrats' Worries About Troy Jackson

TIME - Tech

Follow this section to personalize your feed and get instant alerts. Follow Go to your personalized feed WHY FOLLOW? Smart Alerts: Get notified about major news as it happens. The D.C. Brief Open follow modal Personalized Content Follow this tag to personalize your feed and get instant alerts. Follow Go to your personalized feed WHY FOLLOW? Smart Alerts: Get notified about major news as it happens.


Scaling Laws For Scalable Oversight

Neural Information Processing Systems

Scalable oversight, the process by which weaker AI systems supervise stronger ones, has been proposed as a key strategy to control future superintelligent systems. However, it is still unclear how scalable oversight itself scales. To address this gap, we propose a framework that quantifies the probability of successful oversight as a function of the capabilities of the overseer and the system being overseen. Specifically, our framework models oversight as a game between capability-mismatched players; the players have oversight-specific Elo scores that are a piecewise-linear function of their general intelligence, with two plateaus corresponding to task incompetence and task saturation. We validate our framework with a modified version of the game Nim and then apply it to four oversight games: Mafia, Debate, Backdoor Code and Wargames. For each game, we find scaling laws that approximate how domain performance depends on general AI system capability. We then build on our findings in a theoretical study of Nested Scalable Oversight (NSO), a process in which trusted models oversee untrusted stronger models, which then become the trusted models in the next step. We identify conditions under which NSO succeeds and derive numerically (and in some cases analytically) the optimal number of oversight levels to maximize the probability of oversight success. We also apply our theory to our four oversight games, where we find that NSO success rates at a general Elo gap of 400 are 13.5% for Mafia, 51.7% for Debate, 10.0% for Backdoor Code, and 9.4% for Wargames; these rates decline further when overseeing stronger systems.


Military AINeeds Technically-Informed Regulation to Safeguard AIResearch and its Applications

Neural Information Processing Systems

Military weapon systems and command-and-control infrastructure augmented by artificial intelligence (AI) have seen rapid development and deployment in recent years. However, the sociotechnical impacts of AI on combat systems, military decision-making, and the norms of warfare have been understudied. We focus on a specific subset of lethal autonomous weapon systems (LAWS) that use AI for targeting or battlefield decisions. We refer to this subset as AI-powered lethal autonomous weapon systems (AI-LAWS) and argue that they introduce novel risks--including unanticipated escalation, poor reliability in unfamiliar environments, and erosion of human oversight--all of which threaten both military effectiveness and the openness of AI research. These risks cannot be addressed by high-level policy alone; effective regulation must be grounded in the technical behavior of AI models. We argue that AI researchers must be involved throughout the regulatory lifecycle. Thus, we propose a clear, behavior-based definition of AILAWS--systems that introduce unique risks through their use of modern AI--as a foundation for technically grounded regulation, given that existing frameworks do not distinguish them from conventional LAWS. Using this definition, we propose several technically-informed policy directions and invite greater participation from the AI research community in military AI policy discussions.


1ae5c1db7569a6c2f395020765b119a4-Paper-Position_Paper_Track.pdf

Neural Information Processing Systems

Artificial intelligence (AI) now permeates critical infrastructures and decisionmaking systems where failures produce social, economic, and democratic harm. This position paper challenges the entrenched belief that regulation and innovation are opposites. As evidenced by analogies from aviation, pharmaceuticals, and welfare systems and recent cases of synthetic misinformation, bias and unaccountable decision-making, the absence of well-designed regulation has already created immeasurable damage. Regulation, when thoughtful and adaptive, is not a brake on innovation--it is its foundation. The present position paper examines the EU AIAct as a model of risk-based, responsibility-driven regulation that addresses the Collingridge Dilemma: acting early enough to prevent harm, yet flexibly enough to sustain innovation. Its adaptive mechanisms--regulatory sandboxes, small and medium enterprises (SMEs) support, real-world testing, fundamental rights impact assessment (FRIA)--demonstrate how regulation can accelerate responsibly, rather than delay, technological progress. The position paper summarises how governance tools transform perceived burdens into tangible advantages: legal certainty, consumer trust, and ethical competitiveness.


Scaling Laws For Scalable Oversight

Neural Information Processing Systems

Scalable oversight, the process by which weaker AI systems supervise stronger ones, has been proposed as a key strategy to control future superintelligent systems. However, it is still unclear how scalable oversight itself scales. To address this gap, we propose a framework that quantifies the probability of successful oversight as a function of the capabilities of the overseer and the system being overseen. Specifically, our framework models oversight as a game between capability-mismatched players; the players have oversight-specific Elo scores that are a piecewise-linear function of their general intelligence, with two plateaus corresponding to task incompetence and task saturation. We validate our framework with a modified version of the game Nim and then apply it to four oversight games: Mafia, Debate, Backdoor Code and Wargames. For each game, we find scaling laws that approximate how domain performance depends on general AI system capability. We then build on our findings in a theoretical study of Nested Scalable Oversight (NSO), a process in which trusted models oversee untrusted stronger models, which then become the trusted models in the next step. We identify conditions under which NSO succeeds and derive numerically (and in some cases analytically) the optimal number of oversight levels to maximize the probability of oversight success. We also apply our theory to our four oversight games, where we find that NSO success rates at a general Elo gap of 400 are 13.5\% for Mafia, 51.7\% for Debate, 10.0\% for Backdoor Code, and 9.4\% for Wargames; these rates decline further when overseeing stronger systems.