Regression
Estimating Treatment Effects with Independent Component Analysis
Reizinger, Patrik, Mackey, Lester, Brendel, Wieland, Krishnan, Rahul
The field of causal inference has developed a variety of methods to accurately estimate treatment effects in the presence of nuisance. Meanwhile, the field of identifiability theory has developed methods like Independent Component Analysis (ICA) to identify latent sources and mixing weights from data. While these two research communities have developed largely independently, they aim to achieve similar goals: the accurate and sample-efficient estimation of model parameters. In the partially linear regression (PLR) setting, Mackey et al. (2018) recently found that estimation consistency can be improved with non-Gaussian treatment noise. Non-Gaussianity is also a crucial assumption for identifying latent factors in ICA. We provide the first theoretical and empirical insights into this connection, showing that ICA can be used for causal effect estimation in the PLR model. Surprisingly, we find that linear ICA can accurately estimate multiple treatment effects even in the presence of Gaussian confounders or nonlinear nuisance.
Uncertainty Quantification for Machine Learning-Based Prediction: A Polynomial Chaos Expansion Approach for Joint Model and Input Uncertainty Propagation
Machine learning (ML) surrogate models are increasingly used in engineering analysis and design to replace computationally expensive simulation models, significantly reducing computational cost and accelerating decision-making processes. However, ML predictions contain inherent errors, often estimated as model uncertainty, which is coupled with variability in model inputs. Accurately quantifying and propagating these combined uncertainties is essential for generating reliable engineering predictions. This paper presents a robust framework based on Polynomial Chaos Expansion (PCE) to handle joint input and model uncertainty propagation. While the approach applies broadly to general ML surrogates, we focus on Gaussian Process regression models, which provide explicit predictive distributions for model uncertainty. By transforming all random inputs into a unified standard space, a PCE surrogate model is constructed, allowing efficient and accurate calculation of the mean and standard deviation of the output. The proposed methodology also offers a mechanism for global sensitivity analysis, enabling the accurate quantification of the individual contributions of input variables and ML model uncertainty to the overall output variability. This approach provides a computationally efficient and interpretable framework for comprehensive uncertainty quantification, supporting trustworthy ML predictions in downstream engineering applications.
Sampling from Gaussian Processes: A Tutorial and Applications in Global Sensitivity Analysis and Optimization
Do, Bach, Ajenifuja, Nafeezat A., Adebiyi, Taiwo A., Zhang, Ruda
High-fidelity simulations and physical experiments are essential for engineering analysis and design. However, their high cost often limits their applications in two critical tasks: global sensitivity analysis (GSA) and optimization. This limitation motivates the common use of Gaussian processes (GPs) as proxy regression models to provide uncertainty-aware predictions based on a limited number of high-quality observations. GPs naturally enable efficient sampling strategies that support informed decision-making under uncertainty by extracting information from a subset of possible functions for the model of interest. Despite their popularity in machine learning and statistics communities, sampling from GPs has received little attention in the community of engineering optimization. In this paper, we present the formulation and detailed implementation of two notable sampling methods -- random Fourier features and pathwise conditioning -- for generating posterior samples from GPs. Alternative approaches are briefly described. Importantly, we detail how the generated samples can be applied in GSA, single-objective optimization, and multi-objective optimization. We show successful applications of these sampling methods through a series of numerical examples.
Conformal and kNN Predictive Uncertainty Quantification Algorithms in Metric Spaces
Lugosi, Gรกbor, Matabuena, Marcos
This paper introduces a framework for uncertainty quantification in regression models defined in metric spaces. Leveraging a newly defined notion of homoscedasticity, we develop a conformal prediction algorithm that offers finite-sample coverage guarantees and fast convergence rates of the oracle estimator. In heteroscedastic settings, we forgo these non-asymptotic guarantees to gain statistical efficiency, proposing a local $k$--nearest--neighbor method without conformal calibration that is adaptive to the geometry of each particular nonlinear space. Both procedures work with any regression algorithm and are scalable to large data sets, allowing practitioners to plug in their preferred models and incorporate domain expertise. We prove consistency for the proposed estimators under minimal conditions. Finally, we demonstrate the practical utility of our approach in personalized--medicine applications involving random response objects such as probability distributions and graph Laplacians.
Predictive Process Monitoring Methods: Which One Suits Me Best?
Di Francescomarino, Chiara, Ghidini, Chiara, Maggi, Fabrizio Maria, Milani, Fredrik
Predictive process monitoring has recently gained traction in academia and is maturing also in companies. However, with the growing body of research, it might be daunting for companies to navigate in this domain in order to find, provided certain data, what can be predicted and what methods to use. The main objective of this paper is developing a value-driven framework for classifying existing work on predictive process monitoring. This objective is achieved by systematically identifying, categorizing, and analyzing existing approaches for predictive process monitoring. The review is then used to develop a value-driven framework that can support organizations to navigate in the predictive process monitoring field and help them to find value and exploit the opportunities enabled by these analysis techniques.
Isotonic Quantile Regression Averaging for uncertainty quantification of electricity price forecasts
Lipiecki, Arkadiusz, Uniejewski, Bartosz
--Quantifying the uncertainty of forecasting models is essential to assess and mitigate the risks associated with data-driven decisions, especially in volatile domains such as electricity markets. Machine learning methods can provide highly accurate electricity price forecasts, critical for informing the decisions of market participants. However, these models often lack uncertainty estimates, which limits the ability of decision makers to avoid unnecessary risks. In this paper, we propose a novel method for generating probabilistic forecasts from ensembles of point forecasts, called Isotonic Quantile Regression A veraging (iQRA). Building on the established framework of Quantile Regression A veraging (QRA), we introduce stochastic order constraints to improve forecast accuracy, reliability, and computational costs. In an extensive forecasting study of the German day-ahead electricity market, we show that iQRA consistently outperforms state-of-the-art postprocessing methods in terms of both reliability and sharpness. It produces well-calibrated prediction intervals across multiple confidence levels, providing superior reliability to all benchmark methods, particularly coverage-based conformal prediction. In addition, isotonic regularization decreases the complexity of the quantile regression problem and offers a hyperparameter-free approach to variable selection. The primary goal of a point forecasting model is to provide an accurate prediction of the future value of a variable of interest to aid in the decision making process [1].
Disparities in Peer Review Tone and the Role of Reviewer Anonymity
Sahakyan, Maria, AlShebli, Bedoor
Peer review remains a cornerstone of scholarly publishing, essential for safeguarding the quality, credibility, and integrity of scientific research. Despite its fundamental role, the peer review process is still poorly understood and continues to provoke debate regarding its purpose, effectiveness, and fairness [1]. Growing evidence suggests that peer review is susceptible to social biases that may undermine objectivity and equity in the evaluation of manuscripts [2]. Moreover, recent work highlights systemic shortcomings, including low inter-reviewer agreement, procedural inefficiencies, and limited transparency, which further challenge the integrity of the process [3]. As science becomes increasingly global and interdisciplinary, there is an urgent need to clarify the normative goals of peer review, evaluate alternative models, and develop empirically grounded reforms to mitigate bias and improve the consistency and fairness of scientific evaluation. At its core, peer review is intended to enhance the quality of scientific research by identifying methodological flaws, offering constructive feedback, and flagging potentially misleading claims. However, it has faced persistent criticism for its inefficiencies, lack of transparency, and vulnerability to bias [3, 4, 5, 6, 7]. Despite these concerns, the process continues to receive broad support from researchers and journal stakeholders [8, 9].
Distributed Machine Learning Approach for Low-Latency Localization in Cell-Free Massive MIMO Systems
Kumar, Manish, Chou, Tzu-Hsuan, Lee, Byunghyun, Michelusi, Nicolรฒ, Love, David J., Zhang, Yaguang, Krogmeier, James V.
--Low-latency localization is critical in cellular networks to support real-time applications requiring precise positioning. In this paper, we propose a distributed machine learning (ML) framework for fingerprint-based localization tailored to cell-free massive multiple-input multiple-output (MIMO) systems, an emerging architecture for 6G networks. The proposed framework enables each access point (AP) to independently train a Gaussian process regression model using local angle-of-arrival and received signal strength fingerprints. These models provide probabilistic position estimates for the user equipment (UE), which are then fused by the UE with minimal computational overhead to derive a final location estimate. This decentralized approach eliminates the need for fronthaul communication between the APs and the central processing unit (CPU), thereby reducing latency. Additionally, distributing computational tasks across the APs alleviates the processing burden on the CPU compared to traditional centralized localization schemes. Simulation results demonstrate that the proposed distributed framework achieves localization accuracy comparable to centralized methods, despite lacking the benefits of centralized data aggregation. Moreover, it effectively reduces uncertainty of the location estimates, as evidenced by the 95% covariance ellipse. The results highlight the potential of distributed ML for enabling low-latency, high-accuracy localization in future 6G networks. The next-generation 6G mobile communication is expected to revolutionize wireless communication systems, with integrated sensing and communication (ISAC) playing a key role in enabling advanced connectivity.
Latent Space Data Fusion Outperforms Early Fusion in Multimodal Mental Health Digital Phenotyping Data
Barkat, Youcef, Hamitouche, Dylan, Parekh, Deven, Guo, Ivy, Benrimoh, David
Background: Mental illnesses such as depression and anxiety require improved methods for early detection and personalized intervention. Traditional predictive models often rely on unimodal data or early fusion strategies that fail to capture the complex, multimodal nature of psychiatric data. Advanced integration techniques, such as intermediate (latent space) fusion, may offer better accuracy and clinical utility. Methods: Using data from the BRIGHTEN clinical trial, we evaluated intermediate (latent space) fusion for predicting daily depressive symptoms (PHQ-2 scores). We compared early fusion implemented with a Random Forest (RF) model and intermediate fusion implemented via a Combined Model (CM) using autoencoders and a neural network. The dataset included behavioral (smartphone-based), demographic, and clinical features. Experiments were conducted across multiple temporal splits and data stream combinations. Performance was evaluated using mean squared error (MSE) and coefficient of determination (R2). Results: The CM outperformed both RF and Linear Regression (LR) baselines across all setups, achieving lower MSE (0.4985 vs. 0.5305 with RF) and higher R2 (0.4695 vs. 0.4356). The RF model showed signs of overfitting, with a large gap between training and test performance, while the CM maintained consistent generalization. Performance was best when integrating all data modalities in the CM (in contradistinction to RF), underscoring the value of latent space fusion for capturing non-linear interactions in complex psychiatric datasets. Conclusion: Latent space fusion offers a robust alternative to traditional fusion methods for prediction with multimodal mental health data. Future work should explore model interpretability and individual-level prediction for clinical deployment.
Conformalized Regression for Continuous Bounded Outcomes
Wu, Zhanli, Leisen, Fabrizio, Rubio, F. Javier
Regression problems with bounded continuous outcomes frequently arise in real-world statistical and machine learning applications, such as the analysis of rates and proportions. A central challenge in this setting is predicting a response associated with a new covariate value. Most of the existing statistical and machine learning literature has focused either on point prediction of bounded outcomes or on interval prediction based on asymptotic approximations. We develop conformal prediction intervals for bounded outcomes based on transformation models and beta regression. We introduce tailored non-conformity measures based on residuals that are aligned with the underlying models, and account for the inherent heteroscedasticity in regression settings with bounded outcomes. We present a theoretical result on asymptotic marginal and conditional validity in the context of full conformal prediction, which remains valid under model misspecification. For split conformal prediction, we provide an empirical coverage analysis based on a comprehensive simulation study. The simulation study demonstrates that both methods provide valid finite-sample predictive coverage, including settings with model misspecification. Finally, we demonstrate the practical performance of the proposed conformal prediction intervals on real data and compare them with bootstrap-based alternatives.