Large Language Model
The Download: the tech reshaping IVF and the rise of balcony solar
Plus: After years of insults, Anthropic and SpaceX have teamed up. IVF has brought millions of babies into the world over the last four decades. But the process can still be slow, painful, and expensive--and far from guaranteed to work. Now, a wave of new technologies aims to change that. Researchers are using AI to identify promising sperm and embryos, developing robotic systems that could automate parts of the IVF process, and even exploring controversial genetic editing techniques designed to prevent inherited disease. The technologies could make IVF more effective and accessible.
Perturbation is All You Need for Extrapolating Language Models
Cen, Zetai, Zhu, Jin, Shen, Xinwei, Shi, Chengchun
We introduce a simple yet powerful framework for training large language models. In contrast to the standard autoregressive next-token prediction based on an exact prefix, we propose a perturbation-based procedure that first transforms the prefix into a semantic neighbor and then conditions on this perturbed variant for next-token prediction. This yields a hierarchical model with a pre-post-additive noise structure. Within this framework, we develop a rigorous theory of extrapolability, namely, the capacity of a model class to make reliable predictions for token sequences that lie outside the empirical support of the training corpus. We evaluate the finite-sample performance of the proposed procedure using both synthetic and real-world language data. Results show that the proposed method consistently improves out-of-support prediction while maintaining competitive in-support performance, demonstrating that perturbation offers a practical route to language modeling.
Self-Attention as Transport: Limits of Symmetric Spectral Diagnostics
Dahlem, Dominik, Maniloff, Diego, Misiura, Mac
Large language models hallucinate in predictable ways: attention routing fails by over-concentrating on a narrow set of positions, or by spreading so diffusely that relevance is diluted, and the shape of the failure carries diagnostic signal. A widely used family of spectral methods analyzes the symmetric component of the degree-normalized attention operator, which governs transport capacity; we prove that every transpose-invariant spectral diagnostic of this operator is structurally orientation-blind (it cannot distinguish an operator from its transpose, and therefore cannot detect information-flow direction), with a quantitative converse establishing the asymmetry coefficient $G$ as the unique control parameter for direction. Pairing this with a closed-form bipartite-Cheeger landscape for canonical causal architectures, we show that uniform causal attention satisfies an $n$-independent floor $ϕ\ge 1/5$ with worst cut at $t^\ast/n \approx 0.32$, while window attention pierces the floor as $O(w/n)$; failure modes are shape-different, not just value-different. The resulting two-axis diagnostic ($ϕ$ for capacity, $G$ for direction) yields a falsifiable polarity prediction: bottleneck- and diffuse-dominated benchmarks should exhibit opposite polarity. Under length-controlled evaluation, transport features retain interpretable signal (LC-AUROC from 0.62 to 0.84) on tested models up to 8B parameters, with polarity reversing as predicted between HaluEval and MedHallu.
Elon Musk's Last-Ditch Effort to Control OpenAI: Recruit Sam Altman to Tesla
Messages between Shivon Zilis and Tesla executives reveal plans in 2017 to start a rival AI lab, potentially led by Altman or Demis Hassabis. A few months before Elon Musk left OpenAI's board of directors in February 2018, he tried to recruit Sam Altman to join a "world-class AI lab" within Tesla. Musk went as far as offering the OpenAI CEO a Tesla board seat, according to emails and testimony presented in federal court on Wednesday during the trial . The emails were shown to a jury during the cross examination of Shivon Zilis, a former OpenAI adviser and board member who is also the mother of four of Musk's children. Musk's core claim in this lawsuit is that Altman and OpenAI president Greg Brockman effectively stole a nonprofit, using the $38 million Musk invested to create a private company worth more than $800 billion today.
SpaceX backs Anthropic with data centre deal amidst Musk's OpenAI lawsuit
SpaceX backs Anthropic with data centre deal amidst Musk's OpenAI lawsuit Anthropic has reached a deal to tap the computing resources of Elon Musk's SpaceX, marking a detente with its one-time critic and a boost for both companies in the high-stakes artificial intelligence race. Under the agreement announced on Wednesday, Anthropic will use the full computing power of SpaceX's Colossus 1 facility in Memphis, Tennessee, which houses more than 220,000 Nvidia processors and will give the Claude chatbot maker 300 megawatts of new capacity within a month. That's enough electricity to power more than 300,000 homes - as the Dario Amodei-led company seeks to boost the capacity of its Claude Pro and Claude Max AI assistants for subscribers. The tool allows AI systems to review work between sessions, spot patterns, and update files that store user preferences and other context. Available as a research preview, "dreaming" comes with software for managing agents, or AI programmes that perform tasks with little human involvement.
Canadian officials claim OpenAI violated federal and provincial privacy laws
Philippe Dufresne, the Privacy Commissioner of Canada, has found OpenAI was not compliant with Canadian federal and provincial privacy laws in the training of its AI models. Following an investigation, Dufresne and his counterparts in Alberta, Quebec and British Columbia say OpenAI's approach to things like data collection and consent stepped on multiple laws, including Canada's Personal Information Protection and Electronic Documents Act (PIPEDA), which governs how companies collect and use personal information during the normal course of business. The commissioners participating in the investigation identified multiple privacy issues with OpenAI's approach, including that the company gathered vast amounts of personal information without adequate safeguards to prevent use of that information to train its models, and that it failed to acquire consent to collect and use that personal information in the first place. Warnings in ChatGPT note that interactions with the AI could be used in training, but third-party data OpenAI has purchased or scraped also includes personal details people likely aren't even aware of. The fact that ChatGPT users have no way to access, correct or delete that data was another issue that the commissioners identified, according to a summary of the investigation's findings, along with OpenAI's lackluster attempts to acknowledge the inaccuracy of some of ChatGPT's responses.
Former OpenAI board member says Elon Musk offered her sperm donations
A former OpenAI board member has explained how her unconventional personal relationship with Elon Musk evolved into having four of his children. Shivon Zilis testified in a federal courtroom in Oakland, California for hours on Wednesday as part of Musk's lawsuit trying to reverse OpenAI's change to a for-profit company. The focus of Zilis's appearance was her direct involvement in early talks with Musk around the company becoming a for-profit, but also how she worked for and became involved with Musk as she advised OpenAI. I still really wanted to be a mum and Elon made the offer around that time and I accepted, she said, explaining Musk in 2020 had offered to donate sperm. He was encouraging everyone around him at that time to have kids and he'd noticed I did not.
Anthropic doubles Claude Code limits, thanks to a deal with SpaceX
Anthropic has partnered with SpaceX to double Claude Code usage limits across Pro, Max, Team, and Enterprise plans, according to PCWorld. The deal provides access to SpaceX's Colossus 1 data center featuring over 220,000 Nvidia GPUs, significantly boosting Anthropic's computing capacity. This partnership marks a surprising shift, as Elon Musk previously criticized Anthropic but recently expressed being impressed after meetings with company staff. Instead of downgrading its most affordable Claude subscription plan by dropping access to Claude Code, Anthropic has instead doubled Claude Code usage rates for subscribers, starting today. All it took was an eyebrow-raising alliance with an unlikely partner.
Anthropic Gets in Bed With SpaceX as the AI Race Turns Weird
In an unexpected turn, the two companies signed a deal for Anthropic to use computing resources from Elon Musk's xAI. Anthropic and Elon Musk's SpaceX said on Wednesday that the two entities have signed an agreement for Anthropic to use computing resources from xAI's data center in Memphis, Tennessee. It's the latest tie up in an industry that is scrambling to find enough computers to run complex AI software. SpaceX and xAI were previously separate companies, but the two merged earlier this year. The combined entity, also owned by Musk, is called SpaceXAI.
Google just bought a stake in the maker of Eve Online to train its AI models
The company behind the long-running space sim has entered into a partnership with Google in which the search giant will take a minority stake. In exchange, Google's DeepMind will train its AI technology on the game, according to a report by . CCP Games, the dev who made and maintains, has also been rebranded as Fenris Creations . This happened just after the company purchased the rights to the game back from Korean developer Pearl Abyss. Google's investment is in the millions of dollars, according to Fenris Creations Chief Executive Officer Hilmar Veigar Pétursson.