We introduce a model for neural scaling laws under sparse activations. In the model, test loss is often dominated by rare coordinates that are never observed in the training input. This mechanism induces a novel bottleneck absent from dense models. We derive the asymptotic population loss in both the underparameterized and overparameterized regimes, and show that the loss exhibits a double-descent peak near the interpolation threshold -- where the number of parameters is just sufficient to fit the training data -- resulting in a loss curve governed by two distinct scaling exponents -- one for the overparameterized regime and one for the underparameterized regime -- with a gap determined by the degree of sparsity. Additionally, we derive a compute-optimal frontier that favors increasing dataset size over model capacity under fixed compute budgets. We also analyze gradient-descent dynamics and identify a scaling law for the probability that fixed-step gradient descent becomes unstable. We further show that the sparsity-induced effect persists under nonlinear activations.

We introduce training-free looped transformers, in which a lightweight inference-time wrapper loops a contiguous mid-stack block of layers of a frozen checkpoint without additional fine-tuning, continued training, or architectural changes. Unlike prior looped transformer methods that train with the looped structure end-to-end, we retrofit recurrence onto pretrained models at test time. We show that naive block reapplication usually degrades performance, highlighting the importance of the loop application strategy. Motivated by viewing a pre-norm transformer block as a forward Euler step on an ODE, we instead treat looping as a refinement of the same approximation, replacing one large update with smaller damped sub-steps. Across seven dense, sparse MoE, and MLA+MoE model families, our method improves Qwen3-4B-Instruct by +2.64 pp on MMLU-Pro, Qwen3-30B-A3B-Instruct by +1.14 pp on CommonsenseQA, and Moonlight-16B-A3B-Instruct by +1.20 pp on OpenBookQA.

Facilities can be branded as aligned with Scotland's climate goals despite significant emissions, said APRS. Facilities can be branded as aligned with Scotland's climate goals despite significant emissions, said APRS. Scotland's'green datacentres' policy ignores emissions impact of AI, analysis shows A Scottish government policy designed to encourage datacentres to build in Scotland could lead to a massive volume of carbon emissions being ignored, according to an analysis by a Scottish charity. "Green datacentres" are at the heart of Scotland's ambitions to develop economically. Enshrined in national policy, they are part of a larger, UK-wide effort to attract big AI investment to Scotland.

'Preferred sources' ensures you're seeing the news outlets you want to see. More information Adding us as a Preferred Source in Google by using this link indicates that you would like to see more of our content in Google News results. Get Google News working the way you want it to. Breakthroughs, discoveries, and DIY tips sent six days a week. When you search Google for something topical, you might see a cluster of headlines from news outlets, reporting breaking stories related to your search query.

Figure AI says three humanoid robots named Bob, Frank and Gary ran autonomously for over 24 hours, sorting packages without human control in a warehouse test.

Man finds poop on his roof, and if that wasn't bad enough, it led to a mountain lion encounter Sydney Thomas dominates the red carpet in Cannes as her star continues to rise, new MLB power couple & MEAT! Viral staff photo reveals just how bloated Stephen Colbert's'Late Show' operation really was Four of the most controversial television finales in honor of'The Boys' despised ending Sophie Cuningham has heads spinning with her pregame outfit, Colbert's final jab & lessons from Kyle Busch Adrenaline-packed preview released for upcoming D-Day film'Pressure,' features loaded cast Kacey Musgraves responds to'fat activist' furious because she can't fit into her new Walmart clothing line Selena Gomez is reportedly bringing her talents to award-winning director's new four-hour X-rated movie Minka Kelly uncorks a heater at 45, ABS backfires spectacularly and LSU parents vs a security guard! Robot's lifeless corpse hauled off stage after fall during disastrous Michael Jackson impression Bear cubs spar on woman's front porch in adorable viral nature video, reactions pour in Sen Barrasso details Trump's nearly finalized Iran deal, stance on Strait of Hormuz We must'forget our personal differences' and get back to work: Sen Tommy Tuberville They obviously didn't get the memo here about Memorial Day Weekend being unofficial start of summer. It's cool this morning and it's not even supposed to get into the 80s today. But you know who did receive the memo?

Fake CAPTCHA checks are tricking people into installing malware by asking them to press keyboard shortcuts that execute hidden malicious scripts on their PCs.

Combined net profits at Japanese listed companies in fiscal 2025 rose 9.0% from the previous year to a record, according to SMBC Nikko Securities. Combined net profits at Japanese listed companies in fiscal 2025 rose 9.0% from the previous year to a record ¥54.7 trillion, according to SMBC Nikko Securities. The figure for the year ended in March hit a record high for the fifth consecutive year, thanks to rosy earnings of semiconductor-and data center-related companies amid strong demand for artificial intelligence, as well as strong earnings of banks on the back of higher interest rates. SMBC Nikko Securities compiled earnings data for 1,116 Tokyo Stock Exchange-listed companies in the TOPIX stock price index that close their books in March, with 99.5% of them having disclosed their fiscal 2025 earnings as of Thursday. While auto companies and other transportation equipment firms suffered sharp profit drops from the impact of a U.S. high tariff policy, profits were boosted in a wide range of industry sectors, such as electric appliances, telecommunications and nonferrous metals, supported by growing AI demand.

Health Psychology 3 buttons that don't actually do anything More information Adding us as a Preferred Source in Google by using this link indicates that you would like to see more of our content in Google News results. Buttons can make you feel empowered, even if they don't actually do anything. Breakthroughs, discoveries, and DIY tips sent six days a week. Have you ever pressed a crosswalk button and wondered if it actually does anything? You might be onto something.

At least 82 people have been killed and two are missing after a coal mine blast in northern China, officials have said. The gas explosion at the Liushenyu Coal Mine is the worst mining disaster in China since 2009, and Chinese President Xi Jinping said no effort must be spared in the search and rescue operation. Early on Sunday morning, rescuers deployed mine inspection robots underground, equipped with gas sensors and infrared cameras, state media reported. The BBC's China correspondent Stephen McDonell is at the scene of the blast in Shanxi province. Could a football match soften North Korea-South Korea relations?