Goto

Collaborating Authors

 Technology


Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stability

arXiv.org Machine Learning

This paper studies the stability of low-rank implicit regularization in perturbed deep matrix factorization, where the target matrix is corrupted by a noise matrix. We first derive sufficient spectral conditions under which gradient descent exhibits a low-rank phase in the noiseless setting. These conditions show how the target spectrum, initialization, and step size jointly determine the existence of a nonempty low-rank interval. We then analyze the perturbed gradient descent dynamics, proving convergence guarantees and quantifying how the perturbation affects iteration complexity and eigenvalue recovery. Finally, we show that the low-rank phase persists under perturbation, with explicit dependence on the perturbation size. Numerical experiments support the theoretical findings.


Optimal ridge regularization revisited

arXiv.org Machine Learning

We consider $L^2$-regularized linear (ridge) regression over a finite data sample $X$ with bounded covariance and linear prediction targets $y$ with additive isotropic noise of finite variance. We present an iterative procedure to compute the optimal regularization strength numerically from the generative parameters in the fixed-$X$ setting and prove its convergence at limited noise levels. Our experimental evaluation over synthetic data shows that the proposed procedure combined with sample-based parameter estimates attains near-optimal random-$X$ generalization across a wide range of sample sizes, aspect ratios, and noise levels, at an added computational cost equivalent to one preliminary ridge regression in the underparameterized regime and two in the overparameterized case.


Beyond Lipschitz: Data-Driven Robustness via Discrete Modulus of Continuity

arXiv.org Machine Learning

Robustness of neural networks is commonly quantified via local or global Lipschitz constants. However, Lipschitz continuity can be overly coarse or overly restrictive as global robustness measure, failing to capture nuanced, data-dependent behavior. We propose a data-driven, architecture-agnostic framework based on the discrete modulus of continuity (DMOC), a non linear generalization of Lipschitz continuity that provides a finer notion of robustness. Unlike many existing approaches, DMOC does not require access to model internals and instead evaluates regularity relative to the data distribution. This shifts the focus from the model to the data, which provide a data-driven baseline of regularity against which the network's robustness is assessed. We establish convergence results for DMOC-induced seminorms with explicit data-driven rates in terms of the separation distance, and introduce a scalable minibatch algorithm that reduces the quadratic cost of exact computation, enabling application to large-scale data sets such as ImageNet. Empirically, DMOC serves as an architecture independent diagnostic: it distinguishes trained from untrained networks, reveals underfitting and overfitting regimes, and yields, as a special case, tight Lipschitz estimates comparable to state-of-the-art method such as ECLipsE and ECLipsE-fast.


Deep Neural Networks for Doubly Robust Estimation with Nonprobability Survey Samples

arXiv.org Machine Learning

Integrating probability and nonprobability survey samples is an important problem in modern survey sampling. Nonprobability samples often contain rich outcome information but may lack population representativeness, whereas probability samples provide design-based auxiliary information but may not contain the study variable. We propose a deep neural network (DNN)-assisted doubly robust framework for estimating the finite population mean from these two data sources. The proposed method models the logit sampling score for the nonprobability sample as an unknown nonparametric function and estimates it by maximizing a pseudo-likelihood that combines information from the nonprobability sample and a reference probability sample. The DNN parameters are optimized using the ADAM algorithm. The resulting DNN-estimated sampling scores are incorporated into a DNN-assisted inverse-probability weighted estimator and a deep doubly robust estimator. We establish consistency and convergence rates under regularity conditions and evaluate the finite-sample performance of the proposed estimators through simulation studies and an empirical application using Pew Research Center and Behavioral Risk Factor Surveillance System data. The results suggest that the proposed estimators can improve robustness to parametric propensity-score misspecification, especially when the true selection mechanism is nonlinear.


Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learning

arXiv.org Machine Learning

Many real-world classification tasks require predicting multiple labels per instance, necessitating the optimization of complex evaluation metrics such as the $F$-measure and Jaccard index. While the Empirical Utility Maximization (EUM) framework is natural for these population-level metrics, existing theoretical results are largely limited to asymptotic Bayes-consistency. In this paper, we develop principled learning algorithms for optimizing a broad class of generalized metrics within the EUM framework, grounded in the stronger notion of $H$-consistency. Our key contribution is the design of novel surrogate loss functions for multi-label learning that admit provable $H$-consistency bounds, enabling optimization with non-asymptotic guarantees tailored to the hypothesis class and finite samples. Crucially, we prove these combinatorially formulated surrogates decompose exactly, operating in strictly $O(l)$ time without approximations. Building on this foundation, we introduce MMO (Multi-Label Metric Optimization), a new family of algorithms for optimizing generalized linear-fractional metrics. We validate our approach through extensive experiments, demonstrating robust scalability and superior performance over state-of-the-art continuous baselines on large-scale datasets (MS-COCO, Reuters-21578) in high-sparsity, deep learning regimes. Our results offer both theoretical rigor and practical effectiveness for general multi-label metric optimization.


The General Theory of Localization Methods

arXiv.org Machine Learning

This paper proposes a general machine learning framework called the localization method, which is fundamentally built on two core concepts: localization kernels and local means -- key components that underpin the self-attention mechanism. To establish a rigorous theoretical foundation, the framework is formally defined through two essential pillars: the formulation of the local(-ized) model and the localization trick. We systematically investigate the connections between the localization method and a wide range of existing machine learning models/methods, including (but not limited to) kernel methods, lazy learning, the MeanShift algorithm, relaxation labeling, Hopfield networks, local linear embedding (LLE), fuzzy inference, and denoising autoencoders (DAEs). By dissecting these relationships, we clarify the broader theoretical significance of the localization method and demonstrate its practical applicability across diverse machine learning tasks. Furthermore, we explore advanced extensions of the framework, such as adaptive kernels, hierarchical local models, and non-local models. Notably, we show that the Transformer -- a cornerstone of modern sequence modeling -- can be constructed using hierarchical local models, revealing the ability of the localization method to unify and generalize state-of-the-art architectures. This work not only provides a unified theoretical lens to reinterpret existing models but also offers new methodological tools for designing flexible, data-adaptive learning systems.


We analysed thousands of Trump's posts - here's what we found

BBC News

We analysed thousands of Trump's posts - here's what we found In 2026, Donald Trump's use of social media has escalated. The BBC sifted through thousands of posts on his platform Truth Social to analyse what the President has been saying and when. What was the busiest day? When are the busiest hours? What type of content does President Trump share?


Paul McCartney on playing guitar with Paul Mescal: 'He knew it better than I did!'

BBC News

Paul McCartney on playing guitar with Paul Mescal: 'He knew it better than I did!' Hey, I know you! exclaims Paul McCartney, gripping my hand as we walk into his office in central London. And while I'm realistic enough to know he doesn't really hold treasured memories of our previous encounters, I'm impressed by his ability to defuse the tension of Meeting A Beatle. We gather in Soho at lunchtime. Instead of Wild Honey Pie or Savoy Truffle, McCartney has opted for a simple bagel (topping: a terrifying blend of Marmite and hummus), which he prepared in a kitchenette next to his assistant's desk. As he eats, he scans a printed list of film titles - mainly vintage comedies - looking for something to play at his family movie night.


Google Security Engineer Arrested in Million-Dollar Polymarket Trading Scheme

WIRED

According to federal prosecutors, Michele Spagnuolo made more than $1 million on the prediction market platform using confidential information about Google Search traffic. A Google security engineer has been charged with crimes stemming from allegedly placing trades on Polymarket using confidential internal information from the tech giant. Michele Spagnuolo, a 36-year-old Italian citizen, was arrested this morning in New York, as first reported by ABC News. Spagnuolo is charged with one count each of commodities fraud, wire fraud, and money laundering. He has worked at Google since 2014 and was based out of the company's Zurich, Switzerland, offices.


The world's carmakers are struggling to compete with China

BBC News

The world's carmakers are struggling to compete with China Global carmakers are facing a reckoning as US, European and Japanese brands lose ground to Chinese rivals setting the pace not only in electric vehicles, but also in batteries, design and software. The BBC visited factory floors in Beijing and Hefei on the sidelines of Auto China 2026 - the world's largest car show - and found striking levels of automation and software development speed, leaving foreign brands that once dominated the Chinese market struggling to keep up. We have no chance against this, Honda chief executive Toshihiro Mibe told Japanese media after visiting a highly automated factory in Shanghai. Ford chief executive Jim Farley has also warned that Western carmakers, are in a fight for our lives as Chinese rivals expand globally. After decades spent investing in joint ventures with Chinese partners to build vehicles, foreign carmakers are now changing the nature of those partnerships to stay competitive.