Deep Learning
On Joint Regularization and Calibration in Deep Ensembles
Fredsgaard, Laurits, Schmidt, Mikkel N.
Deep ensembles are a powerful tool in machine learning, improving both model performance and uncertainty calibration. While ensembles are typically formed by training and tuning models individually, evidence suggests that jointly tuning the ensemble can lead to better performance. This paper investigates the impact of jointly tuning weight decay, temperature scaling, and early stopping on both predictive performance and uncertainty quantification. Additionally, we propose a partially overlapping holdout strategy as a practical compromise between enabling joint evaluation and maximizing the use of data for training. Our results demonstrate that jointly tuning the ensemble generally matches or improves performance, with significant variation in effect size across different tasks and metrics. We highlight the trade-offs between individual and joint optimization in deep ensemble training, with the overlapping holdout strategy offering an attractive practical solution. We believe our findings provide valuable insights and guidance for practitioners looking to optimize deep ensemble models.
Generalization in Representation Models via Random Matrix Theory: Application to Recurrent Networks
Moakher, Yessin, Tiomoko, Malik, Louart, Cosme, Liao, Zhenyu
We first study the generalization error of models that use a fixed feature representation (frozen intermediate layers) followed by a trainable readout layer. This setting encompasses a range of architectures, from deep random-feature models to echo-state networks (ESNs) with recurrent dynamics. Working in the high-dimensional regime, we apply Random Matrix Theory to derive a closed-form expression for the asymptotic generalization error. We then apply this analysis to recurrent representations and obtain concise formula that characterize their performance. Surprisingly, we show that a linear ESN is equivalent to ridge regression with an exponentially time-weighted (''memory'') input covariance, revealing a clear inductive bias toward recent inputs. Experiments match predictions: ESNs win in low-sample, short-memory regimes, while ridge prevails with more data or long-range dependencies. Our methodology provides a general framework for analyzing overparameterized models and offers insights into the behavior of deep learning networks.
What Is Adobe Firefly? Here's How to Use This Powerful Generative AI Tool
Adobe Firefly is a deceptively powerful AI playground to generate images, videos, and more. Here's how to make the most of it. All products featured on WIRED are independently selected by our editors. However, we may receive compensation from retailers and/or from purchases of products through these links. Adobe Firefly feels like the best-kept secret in software right now.
AI-powered nimbyism could grind UK planning system to a halt, experts warn
One leading planning lawyer warned such AI services could'supercharge nimbyism'. One leading planning lawyer warned such AI services could'supercharge nimbyism'. Tools that help people scan applications and find grounds for objection have potential to hit government's housebuilding plans The government's plan to use artificial intelligence to accelerate planning for new homes may be about to hit an unexpected roadblock: AI-powered nimbyism. A new service called Objector is offering "policy-backed objections in minutes" to people who are upset about planning applications near their homes. It uses generative AI to scan planning applications and check for grounds for objection, ranking these as "high", "medium" or "low" impact. It then automatically creates objection letters, AI-written speeches to deliver to the planning committees, and even AI-generated videos to "influence councillors".
Left Atrial Segmentation with nnU-Net Using MRI
Hosseinabadi, Fatemeh, Sharifi, Seyedhassan
Accurate segmentation of the left atrium (LA) from cardiac MRI is critical for guiding atrial fibrillation (AF) ablation and constructing biophysical cardiac models. Manual delineation is time-consuming, observer-dependent, and impractical for large-scale or time-sensitive clinical workflows. Deep learning methods, particularly convolutional architectures, have recently demonstrated superior performance in medical image segmentation tasks. In this study, we applied the nnU-Net framework, an automated, self-configuring deep learning segmentation architecture, to the Left Atrial Segmentation Challenge 2013 dataset. The dataset consists of thirty MRI scans with corresponding expert-annotated masks. The nnU-Net model automatically adapted its preprocessing, network configuration, and training pipeline to the characteristics of the MRI data. Model performance was quantitatively evaluated using the Dice similarity coefficient (DSC), and qualitative results were compared against expert segmentations. The proposed nnUNet model achieved a mean Dice score of 93.5, demonstrating high overlap with expert annotations and outperforming several traditional segmentation approaches reported in previous studies. The network exhibited robust generalization across variations in left atrial shape, contrast, and image quality, accurately delineating both the atrial body and proximal pulmonary veins.
Pediatric Appendicitis Detection from Ultrasound Images
Hosseinabadi, Fatemeh, Sharifi, Seyedhassan
Pediatric appendicitis remains one of the most common causes of acute abdominal pain in children, and its diagnosis continues to challenge clinicians due to overlapping symptoms and variable imaging quality. This study aims to develop and evaluate a deep learning model based on a pretrained ResNet architecture for automated detection of appendicitis from ultrasound images. We used the Regensburg Pediatric Appendicitis Dataset, which includes ultrasound scans, laboratory data, and clinical scores from pediatric patients admitted with abdominal pain to Children Hospital. Hedwig in Regensburg, Germany. Each subject had 1 to 15 ultrasound views covering the right lower quadrant, appendix, lymph nodes, and related structures. For the image based classification task, ResNet was fine tuned to distinguish appendicitis from non-appendicitis cases. Images were preprocessed by normalization, resizing, and augmentation to enhance generalization. The proposed ResNet model achieved an overall accuracy of 93.44, precision of 91.53, and recall of 89.8, demonstrating strong performance in identifying appendicitis across heterogeneous ultrasound views. The model effectively learned discriminative spatial features, overcoming challenges posed by low contrast, speckle noise, and anatomical variability in pediatric imaging.
ChatGPT accused of acting as 'suicide coach' in series of US lawsuits
ChatGPT accused of acting as'suicide coach' in series of US lawsuits Chatbot was first used for'general help' with schoolwork or research but'evolved into a psychologically manipulative presence', plaintiffs say ChatGPT has been accused of acting as a "suicide coach" in a series of lawsuits filed this week in California alleging that interactions with the chatbot led to severe mental breakdowns and several deaths. The seven lawsuits include allegations of wrongful death, assisted suicide, involuntary manslaughter, negligence and product liability. Each of the seven plaintiffs initially used ChatGPT for "general help with schoolwork, research, writing, recipes, work, or spiritual guidance", according to a joint statement from the Social Media Victims Law Center and Tech Justice Law Project, which filed the lawsuits in California on Thursday. Over time, however, the chatbot "evolved into a psychologically manipulative presence, positioning itself as a confidant and emotional support", the groups said. "Rather than guiding people toward professional help when they needed it ChatGPT reinforced harmful delusions, and, in some cases, acted as a'suicide coach'."
What Happens When Your Favorite Chatbot Dies?
What Happens When Your Favorite Chatbot Dies? Pillay is an editorial fellow at TIME. Sarah Friar speaks during the Bloomberg New Economy Forum in Singapore on Nov. 18, 2021. Sarah Friar speaks during the Bloomberg New Economy Forum in Singapore on Nov. 18, 2021. Pillay is an editorial fellow at TIME.
High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
Jagannath, Aukosh, Jones-McCormick, Taj, Sarangian, Varnan
We develop a high-dimensional scaling limit for Stochastic Gradient Descent with Polyak Momentum (SGD-M) and adaptive step-sizes. This provides a framework to rigourously compare online SGD with some of its popular variants. We show that the scaling limits of SGD-M coincide with those of online SGD after an appropriate time rescaling and a specific choice of step-size. However, if the step-size is kept the same between the two algorithms, SGD-M will amplify high-dimensional effects, potentially degrading performance relative to online SGD. We demonstrate our framework on two popular learning problems: Spiked Tensor PCA and Single Index Models. In both cases, we also examine online SGD with an adaptive step-size based on normalized gradients. In the high-dimensional regime, this algorithm yields multiple benefits: its dynamics admit fixed points closer to the population minimum and widens the range of admissible step-sizes for which the iterates converge to such solutions. These examples provide a rigorous account, aligning with empirical motivation, of how early preconditioners can stabilize and improve dynamics in settings where online SGD fails.
Forgetting is Everywhere
Sanati, Ben, Lee, Thomas L., McInroe, Trevor, Scannell, Aidan, Malkin, Nikolay, Abel, David, Storkey, Amos
A fundamental challenge in developing general learning algorithms is their tendency to forget past knowledge when adapting to new data. Addressing this problem requires a principled understanding of forgetting; yet, despite decades of study, no unified definition has emerged that provides insights into the underlying dynamics of learning. We propose an algorithm- and task-agnostic theory that characterises forgetting as a lack of self-consistency in a learner's predictive distribution over future experiences, manifesting as a loss of predictive information. Our theory naturally yields a general measure of an algorithm's propensity to forget. To validate the theory, we design a comprehensive set of experiments that span classification, regression, generative modelling, and reinforcement learning. We empirically demonstrate how forgetting is present across all learning settings and plays a significant role in determining learning efficiency. Together, these results establish a principled understanding of forgetting and lay the foundation for analysing and improving the information retention capabilities of general learning algorithms.