ij 1
Accelerating Conformal Prediction via Approximate Leave-One-Out
While conformal prediction provides a general framework for uncertainty quantification in predictive inference, its application is often limited by computational cost. Recent methods, including Jackknife+ and Jackknife-minmax, achieve faster computation by trading a slight loss of efficiency relative to full conformal prediction, but still requires computing leave-one-out refits for all observations. In this paper, we further accelerate conformal prediction by incorporating approximate leave-one-out (ALO) estimators, and establish asymptotic coverage and efficiency. While our proof draws on methods developed for analyzing the consistency of ALO cross-validation risk estimators in high-dimensional statistics, it requires adaptations to handle conformal prediction, where leave-$i$-out residuals are needed for predictions at $x_{n+1}$ rather than just at the training covariate $x_i$. Simulation results validate our theoretical findings, showing that the ALO-based methods achieve coverage and efficiency comparable to the exact methods, while significantly reducing the runtime.
A Proofs
A.2 Proof of proposition 1 Let Pล{B,D,E}, k be a valid kernel (assumptions of theorem 1) with K Inversion of conditional with Bayes rule gives: 'W ลS As a complement, we now explicit the simple forms taken by the posterior limit graph in each case. A.3 Proof of theorem 2 We consider the following hierarchical model, for Nonetheless it can be simplified as we now show. We focus on finding the optimal eigenvectors first. Only the left term in (18) depends on R. The optimization problem for eigenvectors writes: min tr! Note that the identity permutation i.e. for i ล [n], (i) =i is optimal in this case as the ( We will choose this U in what follows as the sign of the axes do not influence the characterization of the final result in Z as a PCA embedding. Note that this solution is not unique if there are repeated eigenvalues.
Kernel Regression in Structured Non-IID Settings: Theory and Implications for Denoising Score Learning
Zhang, Dechen, Shi, Zhenmei, Zhang, Yi, Liang, Yingyu, Zou, Difan
Kernel ridge regression (KRR) is a foundational tool in machine learning, with recent work emphasizing its connections to neural networks. However, existing theory primarily addresses the i.i.d. setting, while real-world data often exhibits structured dependencies - particularly in applications like denoising score learning where multiple noisy observations derive from shared underlying signals. We present the first systematic study of KRR generalization for non-i.i.d. data with signal-noise causal structure, where observations represent different noisy views of common signals. By developing a novel blockwise decomposition method that enables precise concentration analysis for dependent data, we derive excess risk bounds for KRR that explicitly depend on: (1) the kernel spectrum, (2) causal structure parameters, and (3) sampling mechanisms (including relative sample sizes for signals and noises). We further apply our results to denoising score learning, establishing generalization guarantees and providing principled guidance for sampling noisy data points. This work advances KRR theory while providing practical tools for analyzing dependent data in modern machine learning applications.