Goto

Collaborating Authors

 Deep Learning


Bad ChatGPT answer? Maybe you're asking the wrong question

PCWorld

When you purchase through links in our articles, we may earn a small commission. This "meta" prompt makes the AI critique your question and suggest alternatives that might work better. The hardest part about working with ChatGPT, Claude, and Gemini is getting the prompt just right. If you're too specific, the AI may give you a narrow answer that misses the big picture. Or maybe you're asking the model to solve a problem that doesn't actually need fixing.


Amazon Thinks the Future of Data Centers Depends on a Technical Problem It Just Solved

WIRED

The tech giant says a breakthrough in data-center networking has dramatically accelerated the flow of information through its massive cloud infrastructure. Amazon says it recently achieved a major breakthrough in networking design--and has been quietly deploying the new technology in its data centers since late last year. The company claims it has significantly increased data speeds while reducing energy use, potentially giving the tech giant an edge as companies race to build ever-faster systems in the cloud. The new technology hinges on a "quasi-random" design that combines elements of traditional, structured data networks with the performance advantages of more random architectures. Researchers have explored random networks for decades, but the technology has never been successfully scaled.


Are robots nearing their ChatGPT moment? โ€“ podcast

The Guardian

Are robots nearing their ChatGPT moment? Last month at Beijing's half marathon, a robot named Lightning beat the human world record by nearly seven minutes. It's the latest in a string of AI-powered milestones that have got people wondering whether robots are about to enter our everyday lives, just as chatbots have. And the country leading the charge is China, where the government has pledged to invest more than ยฃ100bn in robotics over the next 20 years. To find out how robots are already entering the workforce, and what needs to happen to get them cleaning our homes and weeding our gardens, Ian Sample hears from the Guardian's senior China correspondent, Amy Hawkins, and from Nathan Lepora, professor of robotics and AI at Bristol University, who researches how robots can achieve human-like dexterity


Illinois Lawmakers Just Passed America's Strongest AI Safety Bill

WIRED

Illinois Lawmakers Just Passed America's Strongest AI Safety Bill The bill requires companies like OpenAI, Anthropic, and Google to have third parties confirm they're following safety standards. The Illinois House of Representatives passed a bill on Wednesday requiring frontier AI labs like OpenAI, Anthropic, and Google DeepMind to have their safety practices audited by a third party. If signed into law, AI safety experts tell WIRED, it would be the nation's leading check on the power of major AI companies . The bill, SB 315, now heads to governor JB Pritzker's desk. In a post on social media on Wednesday, Pritzker said he plans to sign the bill, citing a need to hold Big Tech accountable.


GenSBI: Generative Methods for Simulation-Based Inference in JAX

arXiv.org Machine Learning

Flow and diffusion generative models have established themselves as widely adopted density estimators for simulation-based inference (SBI), extending naturally from neural posterior estimation to likelihood and joint density estimation. Their principled optimization objectives and freedom from architectural constraints have driven rapid adoption across the natural sciences. Yet the most widely used SBI libraries remain PyTorch-based, leaving researchers who develop their forward models and analysis pipelines in JAX without a native option. We present GenSBI, an open-source library that implements flow matching, score matching, and denoising diffusion entirely in JAX. The library offers three transformer-based architectures -- SimFormer, Flux1, and a novel Flux1Joint that extends gate-modulated transformer blocks to joint density estimation -- all interchangeable through a unified interface that decouples generative method, neural backbone, and inference mode. GenSBI provides an end-to-end workflow from training through posterior calibration (SBC, TARP, LC2ST) and supports custom architectures with domain-specific embedding networks.


Identifiable Bayesian Deep Generative Copulas with Unknown Layer Widths for Data with Arbitrary Marginal Distributions

arXiv.org Machine Learning

Deep generative models offer powerful tools for multivariate data analysis, but their black-box architectures are often unidentified and difficult to interpret. We introduce the Deep Discrete Encoder (DDE) Copula, an identifiable and interpretable generative model for multivariate data with arbitrary marginal distributions. The model places a hierarchical directed network of binary latent variables inside a copula framework, enabling flexible dependence modeling for mixed discrete and continuous data. Estimation is based on rank likelihoods, which decouple marginal modeling from posterior inference on the DDE parameters and avoid specifying the marginal distributions. We establish conditions for identification of the DDE copula parameters, ensuring that layer-specific parameters provide meaningful summaries of multivariate dependence. We also prove quotient-space posterior consistency for continuous margins under the exact rank likelihood and treat the extended rank likelihood for tied or mixed margins as a generalized likelihood, with concentration under an additional contrast condition. For computation, we propose a stochastic expectation-maximization algorithm for \emph{maximum a posteriori} estimation, together with initialization strategies that improve convergence. To learn network dimension adaptively, we extend Bayesian rank-selection priors to infer layer-specific widths. Simulations show strong finite-sample performance, and a personality-survey analysis reveals interpretable hierarchical latent structure in complex multivariate data.


On the Subgaussianity of Quantized Linear Maps: An AI-Assisted Note

arXiv.org Machine Learning

Simone Bombari asked us whether the 1-bit quantized random vector Y = sgn(Wx) has subgaussian norm bounded by a universal constant. Here W is an n n random Gaussian matrix, and x is an independent standard normal random vector in Rn. The question is nontrivial since the coordinates of Y are not independent. We give a strong positive answer to this question - for any bounded map instead of sgn() - using AI: AIDiscovery and Generalization (Theorem 1): To handle coordinate dependence, Gemini 3.5 Flash1 proposed decomposing the Gaussian vector into independent parts, using one part to "smooth" the sign function, and then applying Gaussian concentration for Lipschitz functions.


Soft Specialists: $ฮฑ$-Rรฉnyi Ensembles for Uncertainty-Aware LLM Post-Training

arXiv.org Machine Learning

Existing training approaches for large language models learn a single set of parameters, based on large volumes of data, which is typically heterogeneous, conflicting and often outright contradictory. As a result, the model is forced to compress conflicting goals, and inherent uncertainties into a single, averaged pattern of behaviour. We propose an $ฮฑ$-Rรฉnyi variational framework for learning distributions over post-training parameters, offering an uncertainty-aware alternative to deep ensemble approaches. The resulting variational objective interpolates between classical variational Bayes and predictively oriented posterior learning, balancing between globally plausible individual models against systems of complementary specialists. We identify local stability criteria, demonstrating how model misspecification can make non-degenerate posterior spread locally favourable, manifesting contradictory or conflicting data as epistemic uncertainty. We apply our framework to LLM post-training, learning an ensemble of LoRA adapters attached to a shared, frozen base model, providing a scalable training procedure for both supervised fine-tuning and preference optimisation. Our approach enables training examples to be softly routed across ensemble members, promoting model specialisation and providing actionable uncertainty estimates across different tasks.


Continual Learning in Modern Hopfield Networks with an Application to Diffusion Models

arXiv.org Machine Learning

Generative models, including diffusion models, are increasingly used as foundation models and adapted through sequential fine-tuning, making continual learning an essential problem setting. However, continual learning in such generative models remains poorly understood: after a task change, what aspects of the learned distribution are most easily lost, and what replay samples should be prioritized? We address these questions through the modern Hopfield energy. Recent links between modern Hopfield networks (MHNs) and diffusion models allow analyses in MHNs to be transferred to diffusion models. We introduce intrinsic forgetting as an increase in Hopfield energy after the task change. In tractable settings in an MHN, we prove that high-energy, outlier-like samples undergo a larger energy increase than cluster-like samples, implying that samples located in sharp, isolated basins are more forgettable. We further analyze memory replay and show that replay is particularly effective for high-energy samples, enabling an energy-based selection of replay samples. We validate these predictions in experiments on MHNs and two diffusion models under continual-learning settings: Stable Diffusion and a pixel-space DDPM. In these diffusion models, Hopfield energy tracks reconstruction-based forgetting, and replay experiments reveal energy-dependent mitigation of forgetting that is consistent with the MHN analysis.


Deep Neural Network Training as Random Effects: An Optimization-Inference Duality

arXiv.org Machine Learning

Deep neural networks (DNNs) have achieved remarkable empirical success, yet their training dynamics remain understood mainly from optimization rather than statistical principles. Here we develop a statistical framework for DNN training in the over-parameterized regime by showing that the prediction induced by continuous-time neural tangent kernel (NTK) gradient flow is exactly equivalent to that from a classical random-effects model. In this framework, training time acts as a variance component, or equivalently an empirical Bayes covariance hyperparameter, governing the allocation of variation from noise to structured signal. This equivalence reveals an optimization-inference duality: the gradient-flow path is both an optimization trajectory and an empirical Bayes random-effects inference path. Conditional on training time, the network output is the posterior mean of the latent signal, and estimating training time by restricted maximum likelihood (REML) turns early stopping into likelihood-based empirical Bayes inference rather than external tuning. This perspective yields a two-stage inferential procedure. First, a variance-component test determines whether DNN training captures statistically significant structure beyond initialization. Second, conditional on training being warranted, REML provides a likelihood-based early stopping rule. The resulting stopping time admits a spectral interpretation in the NTK eigenbasis, where training proceeds until spectral loss decorrelation is achieved. We further establish that REML-guided early stopping achieves asymptotically optimal prediction error for fixed-design in-sample prediction and, under additional random-design regularity conditions, for out-of-sample prediction. This work reframes DNN training as statistical inference and provides a principled foundation for deciding whether and how long to train deep neural networks.