This paper introduces the R package spca, which provides a computational framework for least squares sparse principal component analysis (LS-SPCA). Unlike other SPCA methods, LS-SPCA generates uncorrelated sparse principal components (sPCs) that effectively maximize the explained variance while maintaining strong correlations with standard principal components (PCs). The framework also includes more computationally efficient variants that produce mildly correlated sPCs, which often have lower cardinality while explaining equal or greater variance than the LS-SPCA optimal sPCs. The spca package is built on an efficient C++ backend for matrix computations, with distinct engines for tall and fat matrices, and a flexible R frontend. The user interface offers several options for computing sPCs, such as deciding whether sparsification should stop when a threshold for cumulative variance explained or R2 with the PCs is reached, and choosing between simple forward selection, stepwise forward selection, or backward elimination for variable selection. In addition to the print(), summary(), and plot() methods, the package includes tools for comparing different "spca" solutions, grouping sparse loadings, and representing foreign SPCA solutions as "spca" objects. This article demonstrates with real datasets the use of the package in a typical LS-SPCA workflow and briefly contrasts LS-SPCA with conventional SPCA solutions . Then it compares different LS-SPCA solutions obtained from the dataset. Finally, the performance of spca on large tall and fat matrices is discussed, showing that spca offers a computationally efficient alternative for computing interpretable sPCs.

The Fisher Information Matrix (FIM) is the canonical local measure of the curvature of a statistical model's log-likelihood surface, and its dominant eigenvalue λmax quantifies the worst-case sensitivity of the model's output distribution to infinitesimal parameter perturbation [1, 2]. The spectral properties of the FIM of neural networks have been studied directly in the random matrix theory literature. Pennington and Worah [4] derive the limiting spectral density of the FIM of a single-hidden-layer network in the high-dimensional asymptotic regime, building on the broader programme of analysing neural network Hessian and kernel spectra via random matrix methods [5, 6], with subsequent work extending these techniques to deeper architectures and non-asymptotic regimes [7, 8]. These results characterize the typical (bulk and edge) spectral behaviour of the FIM for a fixed network and a random or structured input ensemble. This paper studies a complementary question, posed as a perturbation problem rather than an asymptotic-spectrum problem: how does the dominant eigenvalue of a fixed, evaluated empirical FIM change under two specific structured perturbations of the underlying distribution? The first perturbation is a change in the conditioning input away from a reference (in-distribution) ensemble. The second is a structured additive perturbation of the model's own parameters by finite-precision quantization noise -- a perturbation of independent mathematical interest, since it falls outside the i.i.d.-input asymptotic regime treated in the random matrix literature cited above, and instead concerns a fixed network whose parameters, not its input distribution, are perturbed by a noise process with a specific, analytically tractable structure (Definition 4.1). To our knowledge, this parameterperturbation question for the FIM's dominant eigenvalue, under either source of departure, has not been previously formalized.

Bogomolny, Bohigas and Schmit (BBS) found that the spectrum of the pairwise distance matrix on N points sampled from a smooth d-dimensional manifold encodes a signature of the underlying geometry. We develop I-BBS (Inference-BBS), a coordinate-free method that identifies a low-dimensional latent sub-manifold embedded in a high-dimensional ambient distance matrix alone, without accessing an ambient high-dimensional vector space. It therefore applies even when that space is only partly observable or undefined. We model the ambient embedding by two classes of generative noise, model-based and model-free. The noise mixes the latent signal with off-manifold components, so the eigenvalues reorganise collectively and the latent geometry cannot be read off eigenvalue by eigenvalue. We recover it instead from two integer-stable signatures that survive the noise: the multiplicity of the top non-Perron multiplet, which fixes $d$, and a parameter-free law for how the multiplet positions shrink as the noise grows. On synthetic spheres $S^1$, $S^2$ and $S^3$ these integer signatures are far more stable under noise than the continuous spectral slope, and a blind test recovers both the manifold and the noise model from a single distance matrix. Applications to neural-network representations and to the dynamic training regime are developed in two companion papers.

Mean Field Variational Inference (MFVI) is widely understood to underestimate posterior variance. By analysing conjugate Bayesian Linear Regression (BLR), we show that this characterization is incomplete: while MFVI underestimates the variance in parameter space, it can overestimate the predictive variance compared to the exact posterior. We show that if the MFVI posterior underestimates predictive variances in some directions, it necessarily overestimates them in others. Crucially, this overestimation occurs in directions where the training data concentrates. This leads to the surprising result that, for a test point drawn from the training distribution, MFVI's expected predictive variance exceeds that of the exact posterior. We demonstrate a pathological case of this effect, where the MFVI posterior fails to reduce predictive variance compared to the prior on in distribution data. We connect these results to the Cold Posterior Effect, arguing that varying the temperature can correct this overestimation, yielding predictions closer to those of the exact posterior. We validate our theory on synthetic and real-world regression tasks.

Multi-Task Learning (MTL) enables a single model to learn multiple tasks simultaneously, leveraging knowledge transfer among tasks for enhanced generalization, and has been widely applied across various domains. However, task imbalance remains a major challenge in MTL. Although balancing the convergence speeds of different tasks is an effective approach to address this issue, it is highly challenging to accurately characterize the training dynamics and convergence speeds of multiple tasks within the complex MTL system. To this end, we attempt to analyze the training dynamics in MTL by leveraging Neural Tangent Kernel (NTK) theory and propose a new MTL method, NTKMTL. Specifically, we introduce an extended NTK matrix for MTL and adopt spectral analysis to balance the convergence speeds of multiple tasks, thereby mitigating task imbalance. Based on the approximation via shared representation, we further propose NTKMTL-SR, achieving training efficiency while maintaining competitive performance. Extensive experiments demonstrate that our methods achieve state-of-the-art performance across a wide range of benchmarks, including both multi-task supervised learning and multi-task reinforcement learning.

We consider the problem of how many samples from a Gaussian multi-index model are required to weakly reconstruct the relevant index subspace. Despite its increasing popularity as a testbed for investigating the computational complexity of neural networks, results beyond the single-index setting remain elusive. In this work, we introduce spectral algorithms based on the linearization of a message passing scheme tailored to this problem. Our main contribution is to show that the proposed methods achieve the optimal reconstruction threshold. Leveraging a high-dimensional characterization of the algorithms, we show that above the critical threshold the leading eigenvector correlates with the relevant index subspace, a phenomenon reminiscent of the Baik-Ben Arous-Peche (BBP) transition in spiked models arising in random matrix theory.

Data attribution methods, which quantify the influence of individual training data points on a machine learning model, have gained increasing popularity in datacentric applications in modern AI. Despite a recent surge of new methods developed in this space, the impact of hyperparameter tuning in these methods remains underexplored. In this work, we present the first large-scale empirical study to understand the hyperparameter sensitivity of common data attribution methods. Our results show that most methods are indeed sensitive to certain key hyperparameters. However, unlike typical machine learning algorithms--whose hyperparameters can be tuned using computationally-cheap validation metrics--evaluating data attribution performance often requires retraining models on subsets of training data, making such metrics prohibitively costly for hyperparameter tuning.

Gradient-based algorithms are a cornerstone of artificial neural network training, yet it remains unclear whether biological neural networks use similar gradientbased strategies during learning. Experiments often discover a diversity of synaptic plasticity rules, but whether these amount to an approximation to gradient descent is unclear. Here we investigate a previously overlooked possibility: that learning dynamics may include fundamentally non-gradient "curl"-like components while still being able to effectively optimize a loss function. Curl terms naturally emerge in networks with inhibitory-excitatory connectivity or Hebbian/anti-Hebbian plasticity, resulting in learning dynamics that cannot be framed as gradient descent on any objective. To investigate the impact of these curl terms, we analyze feedforward networks within an analytically tractable student-teacher framework, systematically introducing non-gradient dynamics through neurons exhibiting rule-flipped plasticity.

What property of the data distribution determines the excess risk of principal component analysis? In this paper, we provide a precise answer to this question. We establish a central limit theorem for the error of the principal subspace estimated by PCA, and derive the asymptotic distribution of its excess risk under the reconstruction loss. We obtain a non-asymptotic upper bound on the excess risk of PCA that recovers, in the large sample limit, our asymptotic characterization. Underlying our contributions is the following result: we prove that the negative block Rayleigh quotient, defined on the Grassmannian, is generalized self-concordant along geodesics emanating from its minimizer of maximum rotation less than π/4.

We study the problem of differentially private second moment estimation and present a new algorithm that achieve strong privacy-utility trade-offs even for worst-case inputs under subsamplability assumptions on the data. We call an input (m,α,β)-subsamplable if a random subsample of size m(or larger) preserves w.p 1 β the spectral structure of the original second moment matrix up to a multiplicative factor of 1 α. Building upon subsamplability, we give a recursive algorithmic framework similar to Kamath et al. (2019) that abides zero-Concentrated Differential Privacy (zCDP) while preserving w.h.p the accuracy of the second moment estimation upto an arbitrary factor of (1 γ). We then show how to apply our algorithm to approximate the second moment matrix of a distribution D, even when a noticeable fraction of the input are outliers.