Adam is a popular and widely used adaptive gradient method in deep learning, which has also received tremendous focus in theoretical research. However, most existing theoretical work primarily analyzes its full-batch version, which differs fundamentally from the stochastic variant used in practice. Unlike SGD, stochastic Adam does not converge to its full-batch counterpart even with infinitesimal learning rates. We present the first theoretical characterization of how batch size affects Adam's generalization, analyzing two-layer over-parameterized CNNs on image data. Our results reveal that while both Adam and AdamW with proper weight decay $λ$ converge to poor test error solutions, their mini-batch variants can achieve near-zero test error. We further prove Adam has a strictly smaller effective weight decay bound than AdamW, theoretically explaining why Adam requires more sensitive $λ$ tuning. Extensive experiments validate our findings, demonstrating the critical role of batch size and weight decay in Adam's generalization performance.

This paper introduces torchsom, an open-source Python library that provides a reference implementation of the Self-Organizing Map (SOM) in PyTorch. This package offers three main features: (i) dimensionality reduction, (ii) clustering, and (iii) friendly data visualization. It relies on a PyTorch backend, enabling (i) fast and efficient training of SOMs through GPU acceleration, and (ii) easy and scalable integrations with PyTorch ecosystem. Moreover, torchsom follows the scikit-learn API for ease of use and extensibility.

Forecasting has undergone a profound transformation in the 21st century, driven by advancements in methodology, computational power, and data availability. The origins of this transformation can be traced back to the late 1990s, when the field of economics began grappling with the challenges and opportunities presented by what is now termed "Big Data"--the availability of large datasets with numerous predictors. This period marked the emergence of systematic efforts to develop tools capable of addressing the high-dimensional nature of these datasets with seminal contributions by Frank Diebold, Mario Forni, Marc Hallin, Marco Lippi, Lucrezia Reichlin, Jim Stock, and Mark Watson (see Reichlin, 2003; Watson, 2003; Diebold, 2003). They laid the foundation for a new era of forecasting, as presented at the World Meeting of the Econometric Society in the summer of 2000 (for a survey see De Mol et al., 2017; Diebold, 2021). Since these early contributions, the field of macroeconomic forecasting has experienced significant progress.

Quantized neural network training optimizes a discrete, non-differentiable objective. The straight-through estimator (STE) enables backpropagation through surrogate gradients and is widely used. While previous studies have primarily focused on the properties of surrogate gradients and their convergence, the influence of quantization hyperparameters, such as bit width and quantization range, on learning dynamics remains largely unexplored. We theoretically show that in the high-dimensional limit, STE dynamics converge to a deterministic ordinary differential equation. This reveals that STE training exhibits a plateau followed by a sharp drop in generalization error, with plateau length depending on the quantization range. A fixed-point analysis quantifies the asymptotic deviation from the unquantized linear model. We also extend analytical techniques for stochastic gradient descent to nonlinear transformations of weights and inputs.

Deep learning based weather forecasting (DLWF) models leverage past weather observations to generate future forecasts, supporting a wide range of downstream tasks, including tropical cyclone (TC) trajectory prediction. In this paper, we investigate their vulnerability to adversarial attacks, where subtle perturbations to the upstream weather forecasts can alter the downstream TC trajectory predictions. Although research on adversarial attacks in DLWF models has grown recently, generating perturbed upstream forecasts that reliably steer downstream output toward attacker-specified trajectories remains a challenge. First, conventional TC detection systems are opaque, non-differentiable black boxes, making standard gradient-based attacks infeasible. Second, the extreme rarity of TC events leads to severe class imbalance problem, making it difficult to develop efficient attack methods that will produce the attacker's target trajectories. Furthermore, maintaining physical consistency in adversarially generated forecasts presents another significant challenge. To overcome these limitations, we propose Cyc-Attack, a novel method that perturbs the upstream forecasts of DLWF models to generate adversarial trajectories. First, we pre-train a differentiable surrogate model to approximate the TC detector's output, enabling the construction of gradient-based attacks. Cyc-Attack also employs skewness-aware loss function with kernel dilation strategy to address the imbalance problem. Finally, a distance-based gradient weighting scheme and regularization are used to constrain the perturbations and eliminate spurious trajectories to ensure the adversarial forecasts are realistic and not easily detectable.

Wasserstein distributionally robust optimization (WDRO) provides a framework for adversarial robustness, yet existing methods based on global Lipschitz continuity or strong duality often yield loose upper bounds or require prohibitive computation. In this work, we address these limitations by introducing a primal approach and adopting a notion of exact Lipschitz certificate to tighten this upper bound of WDRO. In addition, we propose a novel Wasserstein distributional attack (WDA) that directly constructs a candidate for the worst-case distribution. Compared to existing point-wise attack and its variants, our WDA offers greater flexibility in the number and location of attack points. In particular, by leveraging the piecewise-affine structure of ReLU networks on their activation cells, our approach results in an exact tractable characterization of the corresponding WDRO problem. Extensive evaluations demonstrate that our method achieves competitive robust accuracy against state-of-the-art baselines while offering tighter certificates than existing methods. Our code is available at https://github.com/OLab-Repo/WDA

To define a steepest descent method over a neural network, we need to choose a norm for each layer, a way to aggregate these norms across layers, and whether to use normalization. We systematically explore different alternatives for aggregating norms across layers, both formalizing existing combinations of Adam and the recently proposed Muon as a type of non-Euclidean gradient descent, and deriving new variants of the Muon optimizer. Through a comprehensive experimental evaluation of the optimizers within our framework, we find that Muon is sensitive to the choice of learning rate, whereas a new variant we call MuonMax is significantly more robust. We then show how to combine any non-Euclidean gradient method with model based momentum (known as Momo). The new Momo variants of Muon are significantly more robust to hyperparameter tuning, and often achieve a better validation score. Thus for new tasks, where the optimal hyperparameters are not known, we advocate for using Momo in combination with MuonMax to save on costly hyperparameter tuning.

Fingerprinting Radio Frequency (RF) emitters typically involves finding unique emitter characteristics that are featured in their transmitted signals. These fingerprints are nuanced but sufficiently detailed, motivating the pursuit of methods that can successfully extract them. The most granular downstream task is known as Specific Emitter Identification (SEI), which requires a well informed RF fingerprinting (RFF) approach for it to be successful. RFF and SEI have a long history, with numerous application areas in defence and civilian contexts such as signal intelligence, electronic surveillance, physical-layer authentication of wireless communication devices, to name a few. RFF methods also support many other downstream tasks such as Emitter Data Association (EDA) and RF Emitter Clustering (RFEC) and are applicable to a range of transmission types. In recent years, data-driven approaches have become popular in the RFF domain due to their ability to automatically learn intricate fingerprints from raw data. These methods generally deliver superior performance when compared to traditional techniques. The more traditional approaches are often labour-intensive, inflexible and only applicable to a particular emitter type or transmission scheme. Therefore, we consider data-driven Machine Learning (ML)-enabled RFF. In particular, we propose a generic framework for ML-enabled RFF which is inclusive of several popular downstream tasks such as SEI, EDA and RFEC. Each task is formulated as a RF fingerprint-dependent task. A variety of use cases using real RF datasets are presented here to demonstrate the framework for a range of tasks and application areas, such as spaceborne surveillance, signal intelligence and countering drones.

Plus: The Trump administration has laid off thousands of federal health workers. The pendant on Rebecca Jensen-Clem's necklace is composed of 36 silver hexagons entwined in a honeycomb mosaic. At the Keck Observatory, in Hawaii, just as many segments make up a mirror that spans 33 feet, reflecting images of uncharted worlds for her to study. Jensen-Clem, an astronomer at the University of California, Santa Cruz, works with the Keck Observatory to try to detect new planets without leaving our own. It's a pursuit that faces a vast array of obstacles, for example wind, and fluctuations in atmospheric density and temperature. At her lab among the redwoods, Jensen-Clem and her students experiment with new technologies and software to help overcome the challenges, and see into space more clearly.

Whether your chip is running a vintage computer game or the latest DeepSeek model, it'll reward you for speaking its native language. But if you took a look beneath the pixels--the rickety rides, the crowds of hungry, thirsty, barfing people (and the janitors mopping in their wake)--deep down at the level of the code, you saw craftsmanship so obsessive that it bordered on insane. Chris Sawyer, the game's sole developer, wrote the whole thing in assembly. Because if/when the machines take over, we should at least speak their language. Certain programming languages, like Python or Go or C++, are called "high-level" because they work sort of like human language, written in commands and idioms that might fit in at a poetry slam.