Deep Learning
How an intern helped build the AI that shook the world
Chris Maddison was just an intern when he started working on the Go-playing AI that would eventually become AlphaGo. In March 2016, Google DeepMind's artificial intelligence system AlphaGo shocked the world. In a stunning five-match series of Go, the ancient Chinese board game, the AI beat the world's best player, Lee Sedol - a moment that was televised in front of millions and hailed by many as a historic moment in the development of artificial intelligence. Chris Maddison, now a professor of artificial intelligence at the University of Toronto, was then a master's student and helped get the project off the ground. Alex Wilkins: How did the idea for AlphaGo first come about?
The moment that kicked off the AI revolution
Has the technology lived up to its potential? The first time that AlphaGo revealed its full power, it prompted a visceral reaction . Lee Sedol, the world's greatest player of the ancient Chinese board game Go, had grown visibly agitated at the artificial intelligence's prowess. The hushed crowd in downtown Seoul, South Korea, could barely contain its gasps. It was quickly dawning on Lee, and the tens of millions watching at home, that this AI was different to those that had come before. It wasn't just beating Lee, but it was doing so with an almost human-like aptitude.
OpenAI Is Opening the Door to Government Spying
Outside OpenAI's headquarters, a handful of people gathered on Monday holding pieces of colorful chalk. They got down on their knees and started writing messages on the sidewalk. Please no legal mass surveillance. At issue was a business deal that the company recently signed with the Department of Defense, following the Pentagon's sudden turn against Anthropic . OpenAI will now supply its technology to the military for use in classified settings, the sorts that may involve wartime decisions and intelligence-gathering--an agreement, many legal experts told me, that could give the government wide-ranging powers.
The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks
We study the implicit bias of momentum-based optimizers on homogeneous models. We first extend existing results on the implicit bias of steepest descent in homogeneous models to normalized steepest descent with an optional learning rate schedule. We then show that for smooth homogeneous models, momentum steepest descent algorithms like Muon (spectral norm), MomentumGD ($\ell_2$ norm), and Signum ($\ell_\infty$ norm) are approximate steepest descent trajectories under a decaying learning rate schedule, proving that these algorithms too have a bias towards KKT points of the corresponding margin maximization problem. We extend the analysis to Adam (without the stability constant), which maximizes the $\ell_\infty$ margin, and to Muon-Signum and Muon-Adam, which maximize a hybrid norm. Our experiments corroborate the theory and show that the identity of the margin maximized depends on the choice of optimizer. Overall, our results extend earlier lines of work on steepest descent in homogeneous models and momentum-based optimizers in linear models.
Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training
Chen, Wei, Chen, Junle, Wu, Yuqian, Liang, Yuxuan, Zhou, Xiaofang
Spatio-temporal forecasting is fundamental to intelligent systems in transportation, climate science, and urban planning. However, training deep learning models on the massive, often redundant, datasets from these domains presents a significant computational bottleneck. Existing solutions typically focus on optimizing model architectures or optimizers, while overlooking the inherent inefficiency of the training data itself. This conventional approach of iterating over the entire static dataset each epoch wastes considerable resources on easy-to-learn or repetitive samples. In this paper, we explore a novel training-efficiency techniques, namely learning from complexity with dynamic sample pruning, ST-Prune, for spatio-temporal forecasting. Through dynamic sample pruning, we aim to intelligently identify the most informative samples based on the model's real-time learning state, thereby accelerating convergence and improving training efficiency. Extensive experiments conducted on real-world spatio-temporal datasets show that ST-Prune significantly accelerates the training speed while maintaining or even improving the model performance, and it also has scalability and universality.
Sparse Bayesian Deep Functional Learning with Structured Region Selection
Zhu, Xiaoxian, Li, Yingmeng, Ma, Shuangge, Wu, Mengyun
In modern applications such as ECG monitoring, neuroimaging, wearable sensing, and industrial equipment diagnostics, complex and continuously structured data are ubiquitous, presenting both challenges and opportunities for functional data analysis. However, existing methods face a critical trade-off: conventional functional models are limited by linearity, whereas deep learning approaches lack interpretable region selection for sparse effects. To bridge these gaps, we propose a sparse Bayesian functional deep neural network (sBayFDNN). It learns adaptive functional embeddings through a deep Bayesian architecture to capture complex nonlinear relationships, while a structured prior enables interpretable, region-wise selection of influential domains with quantified uncertainty. Theoretically, we establish rigorous approximation error bounds, posterior consistency, and region selection consistency. These results provide the first theoretical guarantees for a Bayesian deep functional model, ensuring its reliability and statistical rigor. Empirically, comprehensive simulations and real-world studies confirm the effectiveness and superiority of sBayFDNN. Crucially, sBayFDNN excels in recognizing intricate dependencies for accurate predictions and more precisely identifies functionally meaningful regions, capabilities fundamentally beyond existing approaches.
Initialization-Aware Score-Based Diffusion Sampling
Fassina, Tiziano, Cardoso, Gabriel, Corff, Sylvan Le, Romary, Thomas
Score-based generative models (SGMs) aim at generating samples from a target distribution by approximating the reverse-time dynamics of a stochastic differential equation. Despite their strong empirical performance, classical samplers initialized from a Gaussian distribution require a long time horizon noising typically inducing a large number of discretization steps and high computational cost. In this work, we present a Kullback-Leibler convergence analysis of Variance Exploding diffusion samplers that highlights the critical role of the backward process initialization. Based on this result, we propose a theoretically grounded sampling strategy that learns the reverse-time initialization, directly minimizing the initialization error. The resulting procedure is independent of the specific score training procedure, network architecture, and discretization scheme. Experiments on toy distributions and benchmark datasets demonstrate competitive or improved generative quality while using significantly fewer sampling steps.
Her husband wanted to use ChatGPT to create sustainable housing. Then it took over his life.
Her husband wanted to use ChatGPT to create sustainable housing. Then it took over his life. Kate Fox says Joe Ceccanti was the'most hopeful person' before he started spending 12 hours a day with a chatbot On 7 August, Kate Fox received a phone call that upended her life. A medical examiner said that her husband, Joe Ceccanti - who had been missing for several hours - had jumped from a railway overpass and died. Ceccanti had no history of depression, she said, nor was he suicidal - he was the "most hopeful person" she had ever known. In fact, according to the witness accounts shared with Fox later, just before Ceccanti jumped, he smiled and yelled: "I'm great!" to the rail yard attendants below when they asked him if he was OK.
Learning Credal Ensembles via Distributionally Robust Optimization
Wang, Kaizheng, Faza, Ghifari Adam, Cuzzolin, Fabio, Chau, Siu Lun, Moens, David, Hallez, Hans
Credal predictors are models that are aware of epistemic uncertainty and produce a convex set of probabilistic predictions. They offer a principled way to quantify predictive epistemic uncertainty (EU) and have been shown to improve model robustness in various settings. However, most state-of-the-art methods mainly define EU as disagreement caused by random training initializations, which mostly reflects sensitivity to optimization randomness rather than uncertainty from deeper sources. To address this, we define EU as disagreement among models trained with varying relaxations of the i.i.d. assumption between training and test data. Based on this idea, we propose CreDRO, which learns an ensemble of plausible models through distributionally robust optimization. As a result, CreDRO captures EU not only from training randomness but also from meaningful disagreement due to potential distribution shifts between training and test data. Empirical results show that CreDRO consistently outperforms existing credal methods on tasks such as out-of-distribution detection across multiple benchmarks and selective classification in medical applications.
Leave big tech behind! How to replace Amazon, Google, X, Meta, Apple – and more
Switching to big tech alternatives is easier than you might imagine. Switching to big tech alternatives is easier than you might imagine. T here's not much to love about big tech these days. So many ills can be laid at its door: social media harms, misinformation, polarisation, mining and misuse of personal data, environmental negligence, tax avoidance, the list goes on. Added to which, Silicon Valley's leaders seem all too keen to cosy up to the Trump administration, to shower the president with bribes - sorry, gifts - and remain silent about his worsening political overreach. And that's before we get to the rampant " enshittification ", as the tech writer Cory Doctorow describes it, which means that by design many big tech products have become less useful and more extractive than they were when we originally signed up to them.