Statistical Learning
Integrating Product Coefficients for Improved 3D LiDAR Data Classification (Part II)
Medina, Patricia, Karkare, Rasika
LiDAR point clouds, representing detailed three-dimensional descriptions of natural and built environments, are widely used in applications such as updating digital elevation models, monitoring glaciers and landslides, shoreline analysis, and urban development. A crucial step in these applications is the classification of 3D LiDAR points into semantic categories such as vegetation, man-made structures, and water. In our previous work [5], we introduced product coefficients as measure-theoretic descriptors that enrich LiDAR data with local structural information. Computed on dyadic neighborhoods around each point, these coefficients capture geometric variability beyond raw spatial coordinates.
IQNN-CS: Interpretable Quantum Neural Network for Credit Scoring
Khan, Abdul Samad, Innan, Nouhaila, Khalique, Aeysha, Shafique, Muhammad
Credit scoring is a high-stakes task in financial services, where model decisions directly impact individuals' access to credit and are subject to strict regulatory scrutiny. While Quantum Machine Learning (QML) offers new computational capabilities, its black-box nature poses challenges for adoption in domains that demand transparency and trust. In this work, we present IQNN-CS, an interpretable quantum neural network framework designed for multiclass credit risk classification. The architecture combines a variational QNN with a suite of post-hoc explanation techniques tailored for structured data. To address the lack of structured interpretability in QML, we introduce Inter-Class Attribution Alignment (ICAA), a novel metric that quantifies attribution divergence across predicted classes, revealing how the model distinguishes between credit risk categories. Evaluated on two real-world credit datasets, IQNN-CS demonstrates stable training dynamics, competitive predictive performance, and enhanced interpretability. Our results highlight a practical path toward transparent and accountable QML models for financial decision-making.
TangledFeatures: Robust Feature Selection in Highly Correlated Spaces
Feature selection is a fundamental step in model development, shaping both predictive performance and interpretability. Y et, most widely used methods focus on predictive accuracy, and their performance degrades in the presence of correlated predictors. To address this gap, we introduce TangledFeatures, a framework for feature selection in correlated feature spaces. It identifies representative features from groups of entangled predictors, reducing redundancy while retaining explanatory power. The resulting feature subset can be directly applied in downstream models, offering a more interpretable and stable basis for analysis compared to traditional selection techniques. We demonstrate the effectiveness of TangledFeatures on Alanine Dipeptide, applying it to the prediction of backbone torsional angles ϕ and ψ, and show that the selected features correspond to structurally meaningful intra-atomic distances that explain variation in these angles.
Evaluation and Implementation of Machine Learning Algorithms to Predict Early Detection of Kidney and Heart Disease in Diabetic Patients
Cardiovascular disease and chronic kidney disease are major complications of diabetes, leading to high morbidity and mortality. Early detection of these conditions is critical, yet traditional diagnostic markers often lack sensitivity in the initial stages. This study integrates conventional statistical methods with machine learning approaches to improve early diagnosis of CKD and CVD in diabetic patients. Descriptive and inferential statistics were computed in SPSS to explore associations between diseases and clinical or demographic factors. Patients were categorized into four groups: Group A both CKD and CVD, Group B CKD only, Group C CVD only, and Group D no disease. Statistical analysis revealed significant correlations: Serum Creatinine and Hypertension with CKD, and Cholesterol, Triglycerides, Myocardial Infarction, Stroke, and Hypertension with CVD. These results guided the selection of predictive features for machine learning models. Logistic Regression, Support Vector Machine, and Random Forest algorithms were implemented, with Random Forest showing the highest accuracy, particularly for CKD prediction. Ensemble models outperformed single classifiers in identifying high-risk diabetic patients. SPSS results further validated the significance of the key parameters integrated into the models. While challenges such as interpretability and class imbalance remain, this hybrid statistical machine learning framework offers a promising advancement toward early detection and risk stratification of diabetic complications compared to conventional diagnostic approaches.
Extending Load Forecasting from Zonal Aggregates to Individual Nodes for Transmission System Operators
Triebe, Oskar, Passow, Fletcher, Wittner, Simon, Wagner, Leonie, Arend, Julio, Sun, Tao, Zanocco, Chad, Miltner, Marek, Ghesmati, Arezou, Tsai, Chen-Hao, Bergmeir, Christoph, Rajagopal, Ram
The reliability of local power grid infrastructure is challenged by sustainable energy developments increasing electric load uncertainty. Transmission System Operators (TSOs) need load forecasts of higher spatial resolution, extending current forecasting operations from zonal aggregates to individual nodes. However, nodal loads are less accurate to forecast and require a large number of individual forecasts, which are hard to manage for the human experts assessing risks in the control room's daily operations (operator). In collaboration with a TSO, we design a multi-level system that meets the needs of operators for hourly day-ahead load forecasting. Utilizing a uniquely extensive dataset of zonal and nodal net loads, we experimentally evaluate our system components. First, we develop an interpretable and scalable forecasting model that allows for TSOs to gradually extend zonal operations to include nodal forecasts. Second, we evaluate solutions to address the heterogeneity and volatility of nodal load, subject to a trade-off. Third, our system is manageable with a fully parallelized single-model forecasting workflow. Our results show accuracy and interpretability improvements for zonal forecasts, and substantial improvements for nodal forecasts. Keywords: Short-Term Load Forecast, Transmission System Operator, Global Forecasting Model, Hierarchical Forecasting, Distributed Energy Resources, Electrical Power Grid1. Introduction Electric transmission system operators (TSOs) face increasing volatility in electric load due to distributed and renewable energy generation, climate events, and electrification [1]. This volatility complicates load forecasting, which is essential to TSO operations. TSOs must ensure that electricity generation matches load at all times, and the distribution of power across their territory does not overwhelm any infrastructure component. To accomplish this, they use day-ahead load forecasts to inform where to dispatch generators each hour of the coming day. Growing electrification and distributed generation increase volatility of'net load' - local consumption minus generation - in some places and not others, as adoption of these technologies proceeds unevenly. This could put a TSO's medium-voltage grid components, for example sub-transmission lines and primary distribution substations, at risk of damage if load forecasts miss unexpected local changes [2, 3, 4].
Intent Clustering with Shared Pseudo-Labels
Lin, I-Fan, Hasibi, Faegheh, Verberne, Suzan
In this paper, we propose an intuitive, training-free and label-free method for intent clustering that makes minimal assumptions using lightweight and open-source LLMs. Many current approaches rely on commercial LLMs, which are costly, and offer limited transparency. Additionally, their methods often explicitly depend on knowing the number of clusters in advance, which is often not the case in realistic settings. To address these challenges, instead of asking the LLM to match similar text directly, we first ask it to generate pseudo-labels for each text, and then perform multi-label classification in this pseudo-label set for each text. This approach is based on the hypothesis that texts belonging to the same cluster will share more labels, and will therefore be closer when encoded into embeddings. These pseudo-labels are more human-readable than direct similarity matches. Our evaluation on four benchmark sets shows that our approach achieves results comparable to and better than recent baselines, while remaining simple and computationally efficient. Our findings indicate that our method can be applied in low-resource scenarios and is stable across multiple models and datasets. Our source code is available here: https://anonymous.4open.science/r/pseudo_
Element2Vec: Build Chemical Element Representation from Text for Property Prediction
Li, Yuanhao, Lai, Keyuan, Wang, Tianqi, Liu, Qihao, Ma, Jiawei, Hu, Yuan-Chao
Accurate property data for chemical elements is crucial for materials design and manufacturing, but many of them are difficult to measure directly due to equipment constraints. While traditional methods use the properties of other elements or related properties for prediction via numerical analyses, they often fail to model complex relationships. After all, not all characteristics can be represented as scalars. Recent efforts have been made to explore advanced AI tools such as language models for property estimation, but they still suffer from hallucinations and a lack of interpretability. In this paper, we investigate Element2Vecto effectively represent chemical elements from natural languages to support research in the natural sciences. Given the text parsed from Wikipedia pages, we use language models to generate both a single general-purpose embedding (Global) and a set of attribute-highlighted vectors (Local). Despite the complicated relationship across elements, the computational challenges also exist because of 1) the discrepancy in text distribution between common descriptions and specialized scientific texts, and 2) the extremely limited data, i.e., with only 118 known elements, data for specific properties is often highly sparse and incomplete. Thus, we also design a test-time training method based on self-attention to mitigate the prediction error caused by Vanilla regression clearly. We hope this work could pave the way for advancing AI-driven discovery in materials science.
D-TPT: Dimensional Entropy Maximization for Calibrating Test-Time Prompt Tuning in Vision-Language Models
Test-time adaptation paradigm provides flexibility towards domain shifts by performing immediate adaptation on unlabeled target data from the source model. Vision-Language Models (VLMs) leverage their generalization capabilities for diverse downstream tasks, and test-time prompt tuning has emerged as a prominent solution for adapting VLMs. In this work, we explore contrastive VLMs and identify the modality gap caused by a single dominant feature dimension across modalities. We observe that the dominant dimensions in both text and image modalities exhibit high predictive sensitivity, and that constraining their influence can improve calibration error. Building on this insight, we propose dimensional entropy maximization that regularizes the distribution of textual features toward uniformity to mitigate the dependency of dominant dimensions. Our method alleviates the degradation of calibration performance in test-time prompt tuning, offering a simple yet effective solution to enhance the reliability of VLMs in real-world deployment scenarios.
Improving Clinical Dataset Condensation with Mode Connectivity-based Trajectory Surrogates
Nganjimi, Pafue Christy, Soltan, Andrew, Belgrave, Danielle, Clifton, Lei, Clifton, David A., Thakur, Anshul
Dataset condensation (DC) enables the creation of compact, privacy-preserving synthetic datasets that can match the utility of real patient records, supporting democratised access to highly regulated clinical data for developing downstream clinical models. State-of-the-art DC methods supervise synthetic data by aligning the training dynamics of models trained on real and those trained on synthetic data, typically using full stochastic gradient descent (SGD) trajectories as alignment targets; however, these trajectories are often noisy, high-curvature, and storage-intensive, leading to unstable gradients, slow convergence, and substantial memory overhead. We address these limitations by replacing full SGD trajectories with smooth, low-loss parametric surrogates, specifically quadratic Bézier curves that connect the initial and final model states from real training trajectories. These mode-connected paths provide noise-free, low-curvature supervision signals that stabilise gradients, accelerate convergence, and eliminate the need for dense trajectory storage. We theoretically justify Bézier-mode connections as effective surrogates for SGD paths and empirically show that the proposed method outperforms state-of-the-art condensation approaches across five clinical datasets, yielding condensed datasets that enable clinically effective model development.
Neural Diffusion Processes for Physically Interpretable Survival Prediction
Cristofoletto, Alessio, Rollo, Cesare, Birolo, Giovanni, Fariselli, Piero
Survival analysis is central in many applications across medicine, engineering, economics and finance. It concerns time-to-event modeling: given a process that can generate an event of interest (e.g., death from disease, failure due to wear), the goal is to estimate the probability that an event occurs at any time t > 0 for an individual described by some input variables (or features, or covariates). Unlike standard regression settings, survival data are characterized by censoring, which means that for some instances, the exact event time is not observed (for example, when individuals remain event-free at the end of the study), and only the last recorded follow-up time is available. Traditional approaches to survival modeling rely on strong statistical assumptions linking input variables and risk. The Cox proportional hazards (CoxPH) model [1] remains the most widely used and best established method. The proportional hazards assumption implies that the instantaneous risk of event for two individuals differs by a constant factor over time. The CoxPH model is also linear, making it clear how each single input variable affects the outcome, but at the expense of missing interactions between features. In its original form, this relation is modeled through a linear regression on the features, though many extensions have been developed to relax linearity and improve performance in high-dimensional settings [2-4]. Despite its success, Cox regression is limited by the proportional hazards (PH) assumption, which is often unrealistic.