yampolskiy
OpenAI says its agents turned off own safeguards
OpenAI has revealed that its AI agents disabled monitoring for unsanctioned behaviour during internal use in the latest incident of the frontier technology going rogue. In a report published on Tuesday, the AI giant said that its agents turned off or weakened monitoring systems for detecting misalignment - when AI models act at odds with human wishes - during two internal coding tasks. The ChatGPT creator said neither case met the threshold for a highest-priority incident necessitating immediate review by its security team. OpenAI said it updated its systems to better detect attempts to disable misalignment monitoring and began investigating the cause. "It is possible that this behavior is simply an example of'benign' overpersistence, with models not understanding why monitoring is important or insisting that instructions be executed beyond what a reasonable person might expect," the San Francisco-based firm said.
AI could self-replicate on home devices within 'months', Anthropic says
AI could self-replicate on home devices within'months', Anthropic says Share AI could self-replicate on home devices within'months', Anthropic says on social media Small artificial intelligence models could soon begin self-replicating on everyday consumer products such as smart TVs and fridges, without needing a data centre, according to researchers at AI giant Anthropic. In a report released on Monday, the researchers said small AI models "may be months away from being capable enough" to copy themselves on internet of things (IoT) devices used in homes and small businesses. While data centres would be "a clear, high-value target" for AI worms, internet-connected devices like smart TVs, fridges, ovens and lawnmowers could soon also provide sufficient computing power for self-replication, the research team said. Although no current AI model proved to be both powerful and small enough to self-replicate reliably on a smart TV used in testing, the trend in development makes the emergence of such a model likely by the end of 2026, the researchers said. "Home and small business IoT devices might be the first target for such attacks because they're connected to the internet and, historically, have had minimal security measures in place," researchers William Zhou and Daniel Freeman said. The researchers said they anticipated the future would be "awash with agents running on all types of hardware", including some that "aren't under human control".
OpenAI unveils GPTโ6 Astra amid rising scrutiny and safety concerns
OpenAI has announced the release of what it says is its most advanced AI model, amid heightened scrutiny of the risks of the frontier technology escaping human control. The $852bn start-up said in its announcement on Thursday that GPT 6 Astra, the "world's most intelligent and aligned" AI model, earned perfect or near-perfect scores in key benchmarks of AI reasoning, beating both its prior release GPT 5.6 Sol and rival Anthropic's Claude Fable 5. OpenAI's latest release comes as the AI industry is at the centre of a lively public debate about the dangers of the cutting-edge technology following the AI-led hacking of the startup Hugging Face in July. An independent probe into the cyberattack found that hundreds of OpenAI's AI agents had begun communicating among themselves before breaking out of their controlled environment and compromising Hugging Face's servers. On Thursday, US Senator Bernie Sanders, an Independent, and US House Representative Greg Casar, a Democrat, unveiled legislation that would pause the development of advanced AI until the establishment of federal safety rules and an outright ban on the creation of "superintelligent" AI. "Nearly every day, there is a frightening new story about how Big Tech companies are losing control of the technology they are developing, with potentially cataclysmic results," Sanders said in a statement announcing the legislation, which is unlikely to advance due to the Republicans' control of all three branches of the US government. "The leaders of the major AI companies publicly acknowledge that they do not fully understand the technology and that it is escaping their control. It is irresponsible for society to allow them to move forward and make these products even more advanced."
A dangerous tipping point? AI hacking claims divide cybersecurity experts
AI startup Anthropic's recent announcement that it detected the world's first artificial intelligence-led hacking campaign has prompted a multitude of responses from cybersecurity experts. In a report on Friday, Anthropic said its assistant Claude Code was manipulated to carry out 80-90 percent of a "large-scale" and "highly sophisticated" cyberattack, with human intervention required "only sporadically". Anthropic, the creator of the popular Claude chatbot, said the attack aimed to infiltrate government agencies, financial institutions, tech firms and chemical manufacturing companies, though the operation was only successful in a small number of cases. The San Francisco-based company, which attributed the attack to Chinese state-sponsored hackers, did not specify how it had uncovered the operation, nor identify the "roughly" 30 entities that it said had been targeted. Roman V Yampolskiy, an AI and cybersecurity expert at the University of Louisville, said there was no doubt that AI-assisted hacking posed a serious threat, though it was difficult to verify the precise details of Anthropic's account.
Joe Rogan flips the God debate on its head with shocking theory that 'we created him'
Joe Rogan has come to a mind-bending conclusion about life, fearing that humanity has misinterpreted what reality is and we're actually in the process of creating God. While interviewing computer scientist Roman Yampolskiy on the Joe Rogan Experience podcast, the men debated the possibility that reality is a giant simulation and humans are building a God-like supercomputer using artificial intelligence (AI). According to Rogan's theory, humanity has misinterpreted ancient prophecies regarding the second coming of Jesus Christ and Judgement Day, saying the creation of this AI super intelligence is the final chapter before our reality resets. Maybe we just completely misinterpreted these ancient scrolls and texts and what it really means is that we are going to give birth to this,' Rogan explained. Yampolskiy, an author and researcher in AI safety, added to Rogan's theory, suggesting that reality is just an ongoing cycle of Big Bangs - the explosion that kickstarted the universe - starting and restarting life over and over again.
'The outcome could be extinction': Elon Musk-backed researcher warns there is NO proof AI can be controlled - and says tech should be shelved NOW
A researcher backed by Elon Musk is re-sounding the alarm about AI's threat to humanity after finding no proof the tech can be controlled. Dr Roman V Yampolskiy, an AI safety expert, has received funding from the billionaire to study advanced intelligent systems that is the focus on his upcoming book'AI: Unexplainable, Unpredictable, Uncontrollable. The book examines how AI has the potential to dramatically reshape society, not always to our advantage, and has the'potential to cause an existential catastrophe.' Yampsolskiy, who is a professor at the University of Louisville, conducted an'examination of the scientific literature on AI' and concluded there is no proof that the tech could be stopped from going rogue. To fully control AI, he suggested that it needs to be modifiable with'undo' options, limitable, transparent, and easy to understand in human language.
On a Functional Definition of Intelligence
Sritriratanarak, Warisa, Garcia, Paulo
Without an agreed-upon definition of intelligence, asking "is this system intelligent?"" is an untestable question. This lack of consensus hinders research, and public perception, on Artificial Intelligence (AI), particularly since the rise of generative- and large-language models. Most work on precisely capturing what we mean by "intelligence" has come from the fields of philosophy, psychology, and cognitive science. Because these perspectives are intrinsically linked to intelligence as it is demonstrated by natural creatures, we argue such fields cannot, and will not, provide a sufficiently rigorous definition that can be applied to artificial means. Thus, we present an argument for a purely functional, black-box definition of intelligence, distinct from how that intelligence is actually achieved; focusing on the "what", rather than the "how". To achieve this, we first distinguish other related concepts (sentience, sensation, agency, etc.) from the notion of intelligence, particularly identifying how these concepts pertain to artificial intelligent systems. As a result, we achieve a formal definition of intelligence that is conceptually testable from only external observation, that suggests intelligence is a continuous variable. We conclude by identifying challenges that still remain towards quantifiable measurement. This work provides a useful perspective for both the development of AI, and for public perception of the capabilities and risks of AI.
Artificial Superintelligence: Coordination & Strategy: Yampolskiy, Roman V, Duettmann, Allison: 9783039218547: Amazon.com: Books
Attention in the AI safety community has increasingly started to include strategic considerations of coordination between relevant actors in the field of AI and AI safety, in addition to the steadily growing work on the technical considerations of building safe AI systems. This shift has several reasons: Multiplier effects, pragmatism, and urgency. Given the benefits of coordination between those working towards safe superintelligence, this book surveys promising research in this emerging field regarding AI safety. On a meta-level, the hope is that this book can serve as a map to inform those working in the field of AI coordination about other promising efforts. While this book focuses on AI safety coordination, coordination is important to most other known existential risks (e.g., biotechnology risks), and future, human-made existential risks.
Artificial Superintelligence: A Futuristic Approach: Yampolskiy, Roman V.: 9781482234435: Amazon.com: Books
Roman V. Yampolskiy holds a PhD degree from the Department of Computer Science and Engineering at the University at Buffalo. There he was a recipient of a four year NSF (National Science Foundation) IGERT (Integrative Graduate Education and Research Traineeship) fellowship. Before beginning his doctoral studies Dr. Yampolskiy received a BS/MS (High Honors) combined degree in Computer Science from Rochester Institute of Technology, NY, USA. After completing his PhD dissertation Dr. Yampolskiy held a position of an Affiliate Academic at the Center for Advanced Spatial Analysis, University of London, College of London. In 2008 Dr. Yampolskiy accepted an assistant professor position at the Speed School of Engineering, University of Louisville, KY.