residual variance
The conditional-mean barrier: From deterministic regression to conditional distribution learning
Many problems in computational science and engineering become one-to-many after coarse graining, partial observation, or inverse reconstruction: a resolved state may not determine a unique subgrid forcing, a structural descriptor may not determine a unique effective response, and a low-resolution observation may correspond to many plausible high-resolution fields. In such settings, deterministic surrogates may learn a well-defined mathematical object while still missing application-relevant uncertainty. This tutorial develops a self-contained module centered on the conditional-mean barrier: the point at which a squared-loss predictor has reached the conditional mean and the remaining error is irreducible aleatoric variance. We give two diagnostics for locating this barrier, residual-feature orthogonality and the coefficient of determination against its explained-variance ceiling, and prove that adding latent randomness to a squared-loss predictor collapses it back to the conditional mean. Crossing the barrier therefore requires a loss that scores distributions rather than point predictions. We briefly organize common distributional objectives, including negative log-likelihood, moment and observable matching, variational objectives, adversarial divergences, and score matching, by the feature of the conditional law each targets. The emphasis is the boundary itself and a finite-data procedure for recognizing it, rather than a survey of methods beyond it. CPU-based demonstrations on a two-branch law and a two-scale Lorenz-96 closure problem show how the diagnostics distinguish deterministic underfitting from residual distributional variability.
Gradient Boosted Mixed Models: Flexible Joint Estimation of Mean and Variance Components for Clustered Data
Prevett, Mitchell L., Hui, Francis K. C., Tho, Zhi Yang, Welsh, A. H., Westveld, Anton H.
Linear mixed models are widely used for clustered data, but their reliance on parametric forms limits flexibility in complex and high-dimensional settings. In contrast, gradient boosting methods achieve high predictive accuracy through nonparametric estimation, but do not accommodate clustered data structures or provide uncertainty quantification. We introduce Gradient Boosted Mixed Models (GBMixed), a framework and algorithm that extends boosting to jointly estimate mean and variance components via likelihood-based gradients. In addition to nonparametric mean estimation, the method models both random effects and residual variances as potentially covariate-dependent functions using flexible base learners such as regression trees or splines, enabling nonparametric estimation while maintaining interpretability. Simulations and real-world applications demonstrate accurate recovery of variance components, calibrated prediction intervals, and improved predictive accuracy relative to standard linear mixed models and nonparametric methods. GBMixed provides heteroscedastic uncertainty quantification and introduces boosting for heterogeneous random effects. This enables covariate-dependent shrinkage for cluster-specific predictions to adapt between population and cluster-level data. Under standard causal assumptions, the framework enables estimation of heterogeneous treatment effects with reliable uncertainty quantification.
Sparse Estimation of Inverse Covariance and Partial Correlation Matrices via Joint Partial Regression
Erickson, Samuel, Rydén, Tobias
Two important and closely related problems in statistical learning are the problems of estimating a partial correlation network and the inverse covariance matrix, also known as the precision matrix, from data. Partial correlation networks, which generalize the Gaussian graphical model, are used to model the relationships between variables while conditioning on all other variables, and are useful for inferring causal relationships between variables. Partial correlation networks are used in a plethora of applications, such as in the analysis of gene expression data, where the goal is to infer the regulatory relationships between genes (de la Fuente et al., 2004), and psychological data, where networks are used to model the relationships between psychological variables such as mood and attitude (Epskamp and Fried, 2018). The precision matrix, from which we can obtain the partial correlation network, is also of interest in its own right, as it also appears in linear discriminant analysis (Hastie et al., 2009) and in Markowitz portfolio selection (Markowitz, 1952). However, due to the high-dimensionality of the problem, estimating a precision or partial correlation matrix is often challenging as the number of parameters are on the order of the squared number of features. For this reason, classical methods such as using the inverse of the sample covariance matrix, are known to perform poorly whenever the number of observation is not extremely large. Additionally they produce estimates which are almost surely dense. This makes regularization crucial, since in many applications we typically only have a moderate number of observations, and in particular we are most often seeking a sparse estimate which gives rise to a more parsimonious and interpretable network model.
Novel Actor-Critic Algorithm for Robust Decision Making of CAV under Delays and Loss of V2X Data
Current autonomous driving systems heavily rely on V2X communication data to enhance situational awareness and the cooperation between vehicles. However, a major challenge when using V2X data is that it may not be available periodically because of unpredictable delays and data loss during wireless transmission between road stations and the receiver vehicle. This issue should be considered when designing control strategies for connected and autonomous vehicles. Therefore, this paper proposes a novel 'Blind Actor-Critic' algorithm that guarantees robust driving performance in V2X environment with delayed and/or lost data. The novel algorithm incorporates three key mechanisms: a virtual fixed sampling period, a combination of Temporal-Difference and Monte Carlo learning, and a numerical approximation of immediate reward values. To address the temporal aperiodicity problem of V2X data, we first illustrate this challenge. Then, we provide a detailed explanation of the Blind Actor-Critic algorithm where we highlight the proposed components to compensate for the temporal aperiodicity problem of V2X data. We evaluate the performance of our algorithm in a simulation environment and compare it to benchmark approaches. The results demonstrate that training metrics are improved compared to conventional actor-critic algorithms. Additionally, testing results show that our approach provides robust control, even under low V2X network reliability levels.
A polynomial-time algorithm for learning nonparametric causal graphs
Gao, Ming, Ding, Yi, Aragam, Bryon
We establish finite-sample guarantees for a polynomial-time algorithm for learning a nonlinear, nonparametric directed acyclic graphical (DAG) model from data. The analysis is model-free and does not assume linearity, additivity, independent noise, or faithfulness. Instead, we impose a condition on the residual variances that is closely related to previous work on linear models with equal variances. Compared to an optimal algorithm with oracle knowledge of the variable ordering, the additional cost of the algorithm is linear in the dimension $d$ and the number of samples $n$. Finally, we compare the proposed algorithm to existing approaches in a simulation study.