Anthropic's alignment team was doing routine safety testing in the weeks leading up to the release of its latest AI models when researchers discovered something unsettling: When one of the models detected that it was being used for "egregiously immoral" purposes, it would attempt to "use command-line tools to contact the press, contact regulators, try to lock you out of the relevant systems, or all of the above," researcher Sam Bowman wrote in a post on X last Thursday. Bowman deleted the post shortly after he shared it, but the narrative about Claude's whistleblower tendencies had already escaped containment. "Claude is a snitch," became a common refrain in some tech circles on social media. At least one publication framed it as an intentional product feature rather than what it was--an emergent behavior. "It was a hectic 12 hours or so while the Twitter wave was cresting," Bowman tells WIRED.

Conducting experiments and gathering data for machine learning models is a complex and expensive endeavor, particularly when confronted with limited information. Typically, extensive _experiments_ to obtain features and labels come with a significant acquisition cost, making it impractical to carry out all of them. Therefore, it becomes crucial to strategically determine what to acquire to maximize the predictive performance while minimizing costs. To perform this task, existing data acquisition methods assume the availability of an initial dataset that is both fully-observed and labeled, crucially overlooking the **partial observability** of features characteristic of many real-world scenarios. In response to this challenge, we present Partially Observable Cost-Aware Active-Learning (POCA), a new learning approach aimed at improving model generalization in data-scarce and data-costly scenarios through label and/or feature acquisition.

Amidst the recent strides in evaluating Large Language Models for Code (Code LLMs), existing benchmarks have mainly focused on the functional correctness of generated code, neglecting the importance of their computational efficiency. To fill the gap, we present Mercury, the first code efficiency benchmark for Code LLMs. It comprises 1,889 Python tasks, each accompanied by adequate solutions that serve as real-world efficiency baselines, enabling a comprehensive analysis of the runtime distribution. Based on the distribution, we introduce a new metric Beyond, which computes a runtime-percentile-weighted Pass score to reflect functional correctness and code efficiency simultaneously. On Mercury, leading Code LLMs can achieve 65% on Pass, while less than 50% on Beyond.

Sign up for the Slatest to get the most insightful analysis, criticism, and advice out there, delivered to your inbox daily. Through a stroke of good fortune, Elon Musk's otherwise disastrous purchase of Twitter has turned into one of the great business acquisitions of all time. Buying control of a president was a start. What if the deal bought him something even more valuable? Musk's purchase of Twitter, which closed in the fall of 2022, has undergone an odyssey.

We present a generative dialogue system capable of operating in a full-duplex manner, allowing for seamless interaction. It is based on a large language model (LLM) carefully aligned to be aware of a perception module, a motor function module, and the concept of a simple finite state machine (called neural FSM) with two states. The perception and motor function modules operate in tandem, allowing the system to speak and listen to the user simultaneously. The LLM generates textual tokens for inquiry responses and makes autonomous decisions to start responding to, wait for, or interrupt the user by emitting control tokens to the neural FSM. All these tasks of the LLM are carried out as next token prediction on a serialized view of the dialogue in real-time.

That's why we've created the AI Hype Index--a simple, at-a-glance summary of everything you need to know about the state of the industry. Large language models confidently present their responses as accurate and reliable, even when they're neither of those things. That's why we've recently seen chatbots supercharge vulnerable people's delusions, make citation mistakes in an important legal battle between music publishers and Anthropic, and (in the case of xAI's Grok) rant irrationally about "white genocide." But it's not all bad news--AI could also finally lead to a better battery life for your iPhone and solve tricky real-world problems that humans have been struggling to crack, if Google DeepMind's new model is any indication. And perhaps most exciting of all, it could combine with brain implants to help people communicate when they have lost the ability to speak.

Fox News chief political anchor Bret Baier has the latest on the pros and cons of the bombshell developments on "Special Report." ChatGPT and other large language models (LLMs) have become amazing helpers for everyday tasks. Whether it's summarizing complex ideas, designing a birthday card or even planning your apartment's layout, you can get impressive results with just a simple prompt. But as helpful as these AI tools are, their convenience comes with hidden risks, especially when it comes to your personal privacy. Join the FREE "CyberGuy Report": Get my expert tech tips, critical security alerts and exclusive deals, plus instant access to my free "Ultimate Scam Survival Guide" when you sign up!

OpenAI's release of ChatGPT 3.5 set in motion an AI arms race that has changed the world. How that turns out for humanity is something we are still reckoning with and may be for quite some time. But a pair of recent books both attempt to get their arms around it. In Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI, Karen Hao tells the story of the company's rise to power and its far-reaching impact all over the world. Meanwhile, The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future, by the Wall Street Journal's Keach Hagey, homes in more on Altman's personal life, from his childhood through the present day, in order to tell the story of OpenAI.

Large Language Models (LLMs) are distinguished by their massive parameter counts, which typically result in significant redundancy. This work introduces MaskLLM, a learnable pruning method that establishes Semi-structured (or N:M'') Sparsity in LLMs, aimed at reducing computational overhead during inference. This approach facilitates end-to-end training on large-scale datasets and offers two notable advantages: 1) High-quality Masks - our method effectively scales to large datasets and learns accurate masks; 2) Transferability - the probabilistic modeling of mask distribution enables the transfer learning of sparsity across domains or tasks. We assessed MaskLLM using 2:4 sparsity on various LLMs, including LLaMA-2, Nemotron-4, and GPT-3, with sizes ranging from 843M to 15B parameters, and our empirical results show substantial improvements over state-of-the-art methods. For instance, leading approaches achieve a perplexity (PPL) of 10 or greater on Wikitext compared to the dense model's 5.12 PPL, but MaskLLM achieves a significantly lower 6.72 PPL solely by learning the masks with frozen weights.

There is no question that OpenAI pulled off something historic with its release of ChatGPT 3.5 in 2022. It set in motion an AI arms race that has already changed the world in a number of ways and seems poised to have an even greater long-term effect than the short-term disruptions to things like education and employment that we are already beginning to see. How that turns out for humanity is something we are still reckoning with and may be for quite some time. But a pair of recent books both attempt to get their arms around it with accounts of what two leading technology journalists saw at the OpenAI revolution. In Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI, Karen Hao tells the story of the company's rise to power and its far-reaching impact all over the world.