Sparse autoencoders (SAEs) are designed to extract interpretable features from language models by enforcing a sparsity constraint. Ideally, training an SAE would yield latents that are both sparse and semantically meaningful. However, many SAE latents activate frequently (i.e., are \emph{dense}), raising concerns that they may be undesirable artifacts of the training procedure. In this work, we systematically investigate the geometry, function, and origin of dense latents and show that they are not only persistent but often reflect meaningful model representations. We first demonstrate that dense latents tend to form antipodal pairs that reconstruct specific directions in the residual stream, and that ablating their subspace suppresses the emergence of new dense features in retrained SAEs -- suggesting that high density features are an intrinsic property of the residual space. We then introduce a taxonomy of dense latents, identifying classes tied to position tracking, context binding, entropy regulation, letter-specific output signals, part-of-speech, and principal component reconstruction. Finally, we analyze how these features evolve across layers, revealing a shift from structural features in early layers, to semantic features in mid layers, and finally to output-oriented signals in the last layers of the model. Our findings indicate that dense latents serve functional roles in language model computation and should not be dismissed as training noise.

Artificial intelligence (AI) systems hold great promise for advancing various scientific disciplines, and are increasingly used in real-world applications. Despite their remarkable progress, further capabilities are expected in order to achieve more general types of intelligence. A critical distinction in this context is between factual knowledge, which can be evaluated against true or false answers (e.g., "what is the capital of England?"), and probabilistic knowledge, reflecting probabilistic properties of the real world (e.g., "what is the sex of a computer science graduate in the US?"). In this paper, our goal is to build a benchmark for understanding the capabilities of LLMs in terms of knowledge of probability distributions describing the real world. Given that LLMs are trained on vast amounts of text, it may be plausible that they internalize aspects of these distributions. Indeed, LLMs are touted as powerful universal approximators of real-world distributions. At the same time, classical results in statistics, known as curse of dimensionality, highlight fundamental challenges in learning distributions in high dimensions, challenging the notion of universal distributional learning. In this work, we develop the first benchmark to directly test this hypothesis, evaluating whether LLMs have access to empirical distributions describing real-world populations across domains such as economics, health, education, and social behavior. Our results demonstrate that LLMs perform poorly overall, and do not seem to internalize real-world statistics naturally. When interpreted in the context of Pearl's Causal Hierarchy (PCH), our benchmark demonstrates that language models do not contain knowledge on observational distributions (Layer 1 of PCH), and thus the Causal Hierarchy Theorem implies that interventional (Layer 2) and counterfactual (Layer 3) knowledge of these models is also limited.

The US government is aiming to ease the pressure on air traffic controllers suffering shutdown-related woes by curtailing flights. But airlines have experience with this kind of sudden disruption. Newark Liberty International Airport is one of the high-traffic airports that could see flight cuts starting Friday. The US Federal Aviation Administration plans to cut 10 percent of flights in 40 high-traffic airports on Friday morning if Congress fails to reopen the federal government by then, Transportation secretary Sean Duffy and FAA chief Bryan Bedford said Wednesday. The announcement came days after the US agency said it faced widespread shortages of air traffic controllers in half of the country's 30 busiest airports and hours-long security lines caused by absences of Transportation Security Administration agents.

Jensen Huang, the head of the world's most valuable company Nvidia, says King Charles III personally handed him a copy of a speech he delivered in 2023 that included a warning about the dangers of artificial intelligence. He said, there's something I want to talk to you about. And he handed me a letter, Huang told the BBC, speaking after receiving the 2025 Queen Elizabeth Prize for Engineering in a ceremony at St James's Palace. The letter was a copy of the speech delivered by the King in 2023 at the world's first AI Summit, held at Bletchley Park . In it the monarch said that the risks of AI needed to be tackled with a sense of urgency, unity and collective strength.

In'watershed moment', Tesla board to vote on Musk's $1 trillion package Tesla's board is set to vote on CEO Elon Musk's $1 trillion pay package as major proxy adviser firms urge shareholders to reject the deal. The vote is scheduled for Thursday and will determine whether Musk secures what is the largest compensation package in corporate history. These firms often influence large passive funds that hold significant stakes in the electric carmaker. Tesla has faced mounting challenges this year, with global sales declining and investor confidence wavering. In July, Tesla reported a 13.5 percent decline in sales in the United States. They jumped 7.4 percent in the third quarter ending in September compared with the same period the year before, as US consumers scrambled to take advantage of a $7,500 EV tax credit that was set to expire that month.

The author of "The Eleventh Hour" looks back on a few works--by Mikhail Bulgakov, Franz Kafka, Voltaire, and E. M. Forster--that have helped him craft his own. Salman Rushdie prefers not to immerse himself in other people's writing when he is working on his own. "When I'm writing fiction, I tend not to read fiction. I actually don't want other people's voices to sneak into my head," Rushdie said recently. That's not to say that other writers' books aren't an important part of his process--posing questions, providing instruction, and offering models of characters.

Microsoft, Alphabet, Meta, and Amazon are investing tens of billions in data centers. AI infrastructure is now a key driver of US economic growth. The amount of capital pouring into AI data center projects is staggering. Last week, Microsoft, Alphabet, Meta, and Amazon reported their 2025 capital expenditures would total roughly $370 billion, and they expect that number to keep rising in 2026. The biggest spender last quarter was Microsoft, which put nearly $35 billion into data centers and other investments, equivalent to 45 percent of its revenue. Rarely, if ever, has a single technology absorbed this much money this quickly.

Doctor's husband who left daughter, 2, to die in hot car while watching adult movies DIES day he was to be locked up I keep hearing the same mortifying whisper about Meghan and Harry... their American dream is about to come crumbling down: MEGYN KELLY Baby girl with horrifying side effects, mom couldn't breathe and dad seriously sick... after simple error turned dream home into a death trap Super hot seductresses entrap Silicon Valley nerds and steal vital secrets... so we tracked one down and found out how Bryan Kohberger's staggering prison slush fund is revealed as prosecutors demand he give money to Idaho victims' families Why screaming female migrant who shouted'Help me, I have papers!' was arrested by ICE at Salt Lake City airport Inside Kate and William's forever home: Princess is kitting out Forest Lodge in her preferred'classic contemporary style' to create a'lovely but absolutely inoffensive' look Somali-American who said protecting illegal migrants from Trump was top priority LOSES bid to become America's wokest mayor Influencer Barbara Jankavski who was known online as'Human Barbie' dies age 31 My girlfriend's new body modification is repulsive. She says she did it for me... This Leftist election landslide was caused by the same vile disease that's triggered a GOP civil war. Famous American writer's son, 19, arrested over alleged plot to bomb Detroit gay bars in ISIS terror attack The murder that haunts the Kennedys: Martha Moxley's loved ones reveal their truth in the FREE The Crime Desk newsletter... as accused cousin cleared in killing breaks cover Taylor Momsen admits Gossip Girl role was'killing' her during'long battle' to quit hit series Inside Zohran Mamdani's woke, celebrity filled victory party after socialist won NYC mayoral election Ariana Grande looks worlds away from her'clean-girl look' as she rocks black curly hair and skinny jeans in a major throwback snap from 18 years ago Three astronauts already in space for six months are now stranded in orbit after their craft may have been damaged by dangerous debris floating around Earth . China's Manned Spaceflight Agency (CMSA) has revealed that crew from its Shenzhou 20 mission will need to stay on board the Chinese station Tiangong.

Breakthroughs, discoveries, and DIY tips sent every weekday. Japan is calling in its army to wrestle its ongoing bear problem . Last month, the country's Ministry of the Environment reported that Asian black bear or moon bear () and brown bear () populations have attacked over 100 people since March. With at least 10 fatalities among the tally, the government announced on November 5 that it is stepping up control efforts by deploying soldiers to Akita prefecture on the island of Honshu in northern Japan. In a statement to reporters, Akita's Governor Kenta Suzuki called the situation "desperate," noting that sightings and attacks are now occurring daily.

Foreign-owned apps secretly collect personal data from users and sell it to overseas data brokers, with retirees being particularly vulnerable to targeted scams.