Goto

Collaborating Authors

 Regression


Efficiently Learning Synthetic Control Models for High-dimensional Disaggregated Data

arXiv.org Machine Learning

The Synthetic Control method (SC) has become a valuable tool for estimating causal effects. Originally designed for single-treated unit scenarios, it has recently found applications in high-dimensional disaggregated settings with multiple treated units. However, challenges in practical implementation and computational efficiency arise in such scenarios. To tackle these challenges, we propose a novel approach that integrates the Multivariate Square-root Lasso method into the synthetic control framework. We rigorously establish the estimation error bounds for fitting the Synthetic Control weights using Multivariate Square-root Lasso, accommodating high-dimensionality and time series dependencies. Additionally, we quantify the estimation error for the Average Treatment Effect on the Treated (ATT). Through simulation studies, we demonstrate that our method offers superior computational efficiency without compromising estimation accuracy. We apply our method to assess the causal impact of COVID-19 Stay-at-Home Orders on the monthly unemployment rate in the United States at the county level.


Beyond Isotonization: Scalable Non-Crossing Quantile Estimation via Neural Networks for Student Growth Percentiles

arXiv.org Machine Learning

Student Growth Percentiles (SGPs), widely adopted across U.S. state assessment systems, employ independent quantile regression followed by post-hoc correction using an isotonic projection method (\texttt{isotonize=TRUE} in the \texttt{SGP} R package) to address quantile crossing. We demonstrate this approach contains a fundamental methodological inconsistency: interpolation between independently-estimated, potentially crossed quantiles requires monotonicity, yet the post-hoc correction alters estimates in ways that may violate the quantile property $P(Y \leq \hat{Q}_ฯ„(Y|X) \mid X) = ฯ„$. We term this the \emph{interpolation paradox}. While theoretically sound constrained joint quantile regression (CJQR) eliminates crossing by enforcing non-crossing constraints during optimization, we analyze its computational complexity (often scaling poorly, e.g., $\mathcal{O}((qn)^3)$ for standard LP solvers) rendering it intractable for large-scale educational data ($n > 100{,}000$). We examine the SGP package's switch to the Frisch-Newton interior point method (\texttt{rq.method.for.large.n="fn"}) for large $N$, noting that while efficient for \emph{independent} QR, it doesn't resolve the joint problem's complexity or the paradox. We propose neural network-based multi-quantile regression (NNQR) with shared hidden layers as a practical alternative. Leveraging the convexity of the composite pinball loss, SGD-based optimization used in NN training can reliably approach the global optimum, offering scalability ($O(n)$) and implicitly reducing crossing. Our empirical analysis shows independent QR yields crossing, while both CJQR and NNQR enforce monotonicity. NNQR emerges as a viable, scalable alternative for operational SGP systems, aligning theoretical validity with computational feasibility.


Deep Gaussian Processes for Functional Maps

arXiv.org Machine Learning

Learning mappings between functional spaces, also known as function-on-function regression, plays a crucial role in functional data analysis and has broad applications, e.g. spatiotemporal forecasting, curve prediction, and climate modeling. Existing approaches, such as functional linear models and neural operators, either fall short of capturing complex nonlinearities or lack reliable uncertainty quantification under noisy, sparse, and irregularly sampled data. To address these issues, we propose Deep Gaussian Processes for Functional Maps (DGPFM). Our method designs a sequence of GP-based linear and nonlinear transformations, leveraging integral transforms of kernels, GP interpolation, and nonlinear activations sampled from GPs. A key insight simplifies implementation: under fixed locations, discrete approximations of kernel integral transforms collapse into direct functional integral transforms, enabling flexible incorporation of various integral transform designs. To achieve scalable probabilistic inference, we use inducing points and whitening transformations to develop a variational learning algorithm. Empirical results on real-world and PDE benchmark datasets demonstrate that the advantage of DGPFM in both predictive performance and uncertainty calibration.


Doubly Robust Estimation of Causal Effects in Strategic Equilibrium Systems

arXiv.org Artificial Intelligence

We introduce the Strategic Doubly Robust (SDR) estimator, a novel framework that integrates strategic equilibrium modeling with doubly robust estimation for causal inference in strategic environments. SDR addresses endogenous treatment assignment arising from strategic agent behavior, maintaining double robustness while incorporating strategic considerations. Theoretical analysis confirms SDR's consistency and asymptotic normality under strategic unconfoundedness. Empirical evaluations demonstrate SDR's superior performance over baseline methods, achieving 7.6\%-29.3\% bias reduction across varying strategic strengths and maintaining robust scalability with agent populations. The framework provides a principled approach for reliable causal inference when agents respond strategically to interventions.


A visual big data system for the prediction of weather-related variables: Jordan-Spain case study

arXiv.org Artificial Intelligence

The Meteorology is a field where huge amounts of data are generated, mainly collected by sensors at weather stations, where different variables can be measured. Those data have some particularities such as high volume and dimensionality, the frequent existence of missing values in some stations, and the high correlation between collected variables. In this regard, it is crucial to make use of Big Data and Data Mining techniques to deal with those data and extract useful knowledge from them that can be used, for instance, to predict weather phenomena. In this paper, we propose a visual big data system that is designed to deal with high amounts of weather-related data and lets the user analyze those data to perform predictive tasks over the considered variables (temperature and rainfall). The proposed system collects open data and loads them onto a local NoSQL database fusing them at different levels of temporal and spatial aggregation in order to perform a predictive analysis using univariate and multivariate approaches as well as forecasting based on training data from neighbor stations in cases with high rates of missing values. The system has been assessed in terms of usability and predictive performance, obtaining an overall normalized mean squared error value of 0.00013, and an overall directional symmetry value of nearly 0.84. Our system has been rated positively by a group of experts in the area (all aspects of the system except graphic desing were rated 3 or above in a 1-5 scale). The promising preliminary results obtained demonstrate the validity of our system and invite us to keep working on this area.


Interpret Policies in Deep Reinforcement Learning using SILVER with RL-Guided Labeling: A Model-level Approach to High-dimensional and Multi-action Environments

arXiv.org Artificial Intelligence

Deep reinforcement learning (RL) achieves remarkable performance but lacks interpretability, limiting trust in policy behavior. The existing SIL VER framework (Li, Siddique, and Cao 2025) explains RL policy via Shapley-based regression but remains restricted to low-dimensional, binary-action domains. We propose SIL VER with RL-guided labeling, an enhanced variant that extends SIL VER to multi-action and high-dimensional environments by incorporating the RL policy's own action outputs into the boundary points identification. Our method first extracts compact feature representations from image observations, performs SHAP-based feature attribution, and then employs RL-guided labeling to generate behaviorally consistent boundary datasets. Surrogate models, such as decision trees and regression-based functions, are subsequently trained to interpret RL policy's decision structure. We evaluate the proposed framework on two Atari environments using three deep RL algorithms and conduct human-subject study to assess the clarity and trustworthiness of the derived interpretable policy. Results show that our approach maintains competitive task performance while substantially improving transparency and human understanding of agent behavior.


Quantitative LLM Judges

arXiv.org Artificial Intelligence

LLM-as-a-judge is a framework where a large language model (LLM) evaluates the output of another LLM. While LLMs excel at producing qualitative textual evaluations, they often struggle to predict human preferences and numeric scores. We propose quantitative LLM judges, which align evaluation scores of existing LLM judges to humans in a given domain using regression models. The models are trained to improve the score of the original judge using its rationale and score. We present four quantitative judges for different types of absolute and relative feedback, which showcases the generality and versatility of our framework. Our framework is more computationally efficient than supervised fine-tuning and can be more statistically efficient when human feedback is limited, which is expected in practice. We validate these claims empirically on four datasets using two base judges. Our experiments show that quantitative judges can improve the predictive power of existing judges through post-hoc modeling.


Wasserstein Transfer Learning

arXiv.org Artificial Intelligence

Transfer learning is a powerful paradigm for leveraging knowledge from source domains to enhance learning in a target domain. However, traditional transfer learning approaches often focus on scalar or multivariate data within Euclidean spaces, limiting their applicability to complex data structures such as probability distributions. To address this limitation, we introduce a novel transfer learning framework for regression models whose outputs are probability distributions residing in the Wasserstein space. When the informative subset of transferable source domains is known, we propose an estimator with provable asymptotic convergence rates, quantifying the impact of domain similarity on transfer efficiency. For cases where the informative subset is unknown, we develop a data-driven transfer learning procedure designed to mitigate negative transfer. The proposed methods are supported by rigorous theoretical analysis and are validated through extensive simulations and real-world applications. The code is available at https://github.com/h7nian/WaTL


Understanding Mechanistic Role of Structural and Functional Connectivity in Tau Propagation Through Multi-Layer Modeling

arXiv.org Artificial Intelligence

Alzheimer's disease (AD) is a progressive neurodegenerative disorder marked by the pathological accumulation and propagation of tau proteins [1]. Tau pathology, initially localized in the entorhinal cortex, gradually spreads to connected brain regions in a pattern that correlates with cognitive decline and disease severity. Mounting evidence supports the hypothesis that this spread occurs in a prion-like manner, whereby misfolded tau seeds propagate trans-synaptically through neural networks [2]. This insight has shifted the focus from region-specific atrophy to network-based degeneration, assuming the brain's large-scale connectivity architecture as a key determinant of pathological progression. In this regard, a growing body of research has developed connectome-based diffusion models to simulate and predict the spatial and temporal dynamics of tau propagation across the brain [3-5]. These models typically leverage information from structural connectivity (SC), derived from diffusion-weighted imaging (DWI), or functional connectivity (FC), measured via resting-state functional magnetic resonance imaging (fMRI), to model tau spread as a network-driven diffusion process. By modeling tau dynamics within the topology of the brain connectome, these frameworks offer a mechanistic perspective on how pathological burden evolves across regions. Given the connectome-constrained assumption on tau propagation, such models have shown potential in forecasting future tau accumulation [3, 5], identifying vulnerable brain circuits [5], and stratifying individuals based on progression risk [6], thereby informing early diagnosis and therapeutic targeting.


Are Greedy Task Orderings Better Than Random in Continual Linear Regression?

arXiv.org Artificial Intelligence

We analyze task orderings in continual learning for linear regression, assuming joint realizability of training data. We focus on orderings that greedily maximize dissimilarity between consecutive tasks, a concept briefly explored in prior work but still surrounded by open questions. Using tools from the Kaczmarz method literature, we formalize such orderings and develop geometric and algebraic intuitions around them. Empirically, we demonstrate that greedy orderings converge faster than random ones in terms of the average loss across tasks, both for linear regression with random data and for linear probing on CIFAR-100 classification tasks. Analytically, in a high-rank regression setting, we prove a loss bound for greedy orderings analogous to that of random ones. However, under general rank, we establish a repetition-dependent separation. Specifically, while prior work showed that for random orderings, with or without replacement, the average loss after $k$ iterations is bounded by $\mathcal{O}(1/\sqrt{k})$, we prove that single-pass greedy orderings may fail catastrophically, whereas those allowing repetition converge at rate $\mathcal{O}(1/\sqrt[3]{k})$. Overall, we reveal nuances within and between greedy and random orderings.