Goto

Collaborating Authors

 bottleneck


Roundtables: Inside the "Censorship-Industrial Complex" Idea Shaping US Policy

MIT Technology Review

Watch a subscriber-only conversation to learn about what this idea means for the future of American democracy and the internet. The "censorship-industrial complex" is an idea that a network of government, tech, and research groups is collaborating to suppress conservative online speech. This was fodder for the right-wing information sphere for years--then it began making its way into US policy. Watch a conversation exploring how it started, where it's going, and what it means for the future of democracy and the internet. A startup claims it broke through a bottleneck that's holding back LLMs Will Douglas Heaven A startup claims it broke through a bottleneck that's holding back LLMs Subquadratic has now shared more details about its new model. But some are still skeptical.


The Download: the next big thing in LLMs and how AI academic research is shifting

MIT Technology Review

Plus: Nvidia has secured $500 billion from Wall Street for AI infrastructure. Nine years after Google researchers introduced the transformer, this family of neural networks has become the engine inside every major large language model. But transformers are starting to show their age. As LLMs get bigger and better, transformers have become a bottleneck. Their dense attention mechanism becomes increasingly expensive as the amount of text grows, and they're not great at keeping track of a lot of information at once. Here are four new ideas for how to solve the transformer problem --innovations that could change LLMs for good, making them faster, far more efficient, and (maybe) even smarter.


Puzzle Corner

MIT Technology Review

Click here for the September/October 2026 Puzzle Corner, brought to you by Michael S. Branicky, ScD '95, of the Puzzle Corner Puzzle Crew (aka PC), which also includes Edward Faulkner '03, MEng '04, and Abe Kunin '03. This column includes solutions to the May/June issue. Editor emeritus Allan Gottlieb '67 launched Puzzle Corner in 1966. A startup claims it broke through a bottleneck that's holding back LLMs Will Douglas Heaven The "steroid olympics" were a circus--and a window into our culture Amit Katwala A startup claims it broke through a bottleneck that's holding back LLMs Subquadratic has now shared more details about its new model. But some are still skeptical. The "steroid olympics" were a circus--and a window into our culture Dozens of athletes on performance-enhancing drugs competed in the Enhanced Games.


How AI helps scientists design the next generation of medicines

MIT Technology Review

As generative AI captures public attention, a different kind of AI is reshaping drug discovery. Machine learning models are helping to compress decade-long timelines and cracking problems that were previously unsolvable. Designing and developing a new medicine is an expensive, failure-prone scientific challenge. A new drug can take many years to develop, at the cost of a significant investment. And even then, most possible candidates never reach the patient. For biologic medicines, therapies made from engineered proteins rather than synthetic chemistry (which are often used to treat conditions across most major acute and chronic diseases), the complexity is even greater.


Achieving operational excellence with AI

MIT Technology Review

As AI reshapes how work gets done, organizations with strong process frameworks are best positioned to lead and maintain operational rigor at scale. Frameworks like Lean Six Sigma and business process management (BPM) first gained traction because they promised clarity in the chaos--a structured way to bring order to messy, sprawling operations. Lean Six Sigma emphasized statistical rigor and quality control; BPM created end-to-end maps of how work should flow across departments. Both offered a repeatable way to embed habits of measurement, analysis, and accountability into day-to-day company culture. But today, those time-tested playbooks are evolving as companies seek to embed AI into established process excellence methodologies. By some estimates, the market for AI-powered process optimization is projected to exceed $113 billion within the next decade.


Deeper with Riemannian Geometry: Overcoming Oversmoothing and Oversquashing for Graph Foundation Models

Neural Information Processing Systems

Message Passing Neural Networks (MPNNs) are the building block of graph foundation models, but fundamentally suffer from oversmoothing and oversquashing. There has recently been a surge of interest in fixing both issues. Existing efforts primarily adopt global approaches, which may be beneficial in some regions but detrimental in others, ultimately leading to the suboptimal expressiveness. In this paper, we begin by revisiting oversquashing through a global measure - spectral gap λ- and prove that the increase of λleads to gradient vanishing with respect to the input features, thereby undermining the effectiveness of message passing. Motivated by such theoretical insights, we propose a local approach that adaptively adjusts message passing based on local structures. To achieve this, we connect local Riemannian geometry with MPNNs, and establish a novel nonhomogeneous boundary condition to address both oversquashing and oversmoothing. Building on the Robin condition, we design a GBN network with local bottleneck adjustment, coupled with theoretical guarantees. Extensive experiments on homophilic and heterophilic graphs show the expressiveness of GBN. Furthermore, GBN does not exhibit performance degradation even when the network depth exceeds 256 layers.


The Download: AI bottleneck debates, and BCI trials take off

MIT Technology Review

Plus: Amazon workers who backed data center limits face potential termination. A startup claims it broke through a bottleneck that's holding back LLMs AI startup Subquadratic came out of stealth last month with a huge claim: it had solved a mathematical bottleneck that had held back large language models for almost a decade. The purported breakthrough comes from slashing the number of computations transformers need to carry out to generate answers. The result is a faster and cheaper LLM that uses far less energy than any other model on the market. Many experts remained skeptical--but Subquadratic has started to share the receipts. They suggest that their approach might be worth paying attention to.


A startup claims it broke through a bottleneck that's holding back LLMs

MIT Technology Review

Miami-based AI startup Subquadratic came out of stealth mode last month with a huge claim. It announced that it had solved a mathematical bottleneck that had been holding back large language models for almost a decade. The details were thin, and many people were unconvinced. But Subquadratic has started to bring the receipts, sharing the results of an independent evaluation of its new tech. The results suggest that the company's claims might be worth paying attention to.


Mind the GAP! The Challenges of Scale in Pixel-based Deep Reinforcement Learning

Neural Information Processing Systems

Scaling deep reinforcement learning in pixel-based environments presents a significant challenge, often resulting in diminished performance. While recent works have proposed algorithmic and architectural approaches to address this, the underlying cause of the performance drop remains unclear.


Reduction-based Pseudo-label Generation for Instance-dependent Partial Label Learning

Neural Information Processing Systems

Instance-dependent Partial Label Learning (ID-PLL) aims to learn a multi-class predictive model given training instances annotated with candidate labels related to features, among which correct labels are hidden fixed but unknown. The previous works involve leveraging the identification capability of the training model itself to iteratively refine supervision information. However, these methods overlook a critical aspect of ID-PLL: within the original label space, the model may fail to distinguish some incorrect candidate labels that are strongly correlated with features from correct labels. This leads to poor-quality supervision signals and creates a bottleneck in the training process. In this paper, we propose to leverage reduction-based pseudo-labels to alleviate the influence of incorrect candidate labels and train our predictive model to overcome this bottleneck. Specifically, reduction-based pseudo-labels are generated by performing weighted aggregation on the outputs of a multi-branch auxiliary model, with each branch trained in a label subspace that excludes certain labels. This approach ensures that each branch explicitly avoids the disturbance of the excluded labels, allowing the pseudo-labels provided for instances troubled by these excluded labels to benefit from the unaffected branches. Theoretically, we demonstrate that reduction-based pseudo-labels exhibit greater consistency with the Bayes optimal classifier compared to pseudo-labels directly generated from the training predictive model.