Statistical Learning
RAGFormer: Learning Semantic Attributes and Topological Structure for Fraud Detection
Li, Haolin, Jiang, Shuyang, Zhang, Lifeng, Du, Siyuan, Ye, Guangnan, Chai, Hongfeng
Fraud detection remains a challenging task due to the complex and deceptive nature of fraudulent activities. Current approaches primarily concentrate on learning only one perspective of the graph: either the topological structure of the graph or the attributes of individual nodes. However, we conduct empirical studies to reveal that these two types of features, while nearly orthogonal, are each independently effective. As a result, previous methods can not fully capture the comprehensive characteristics of the fraud graph. To address this dilemma, we present a novel framework called Relation-Aware GNN with transFormer~(RAGFormer) which simultaneously embeds both semantic and topological features into a target node. The simple yet effective network consists of a semantic encoder, a topology encoder, and an attention fusion module. The semantic encoder utilizes Transformer to learn semantic features and node interactions across different relations. We introduce Relation-Aware GNN as the topology encoder to learn topological features and node interactions within each relation. These two complementary features are interleaved through an attention fusion module to support prediction by both orthogonal features. Extensive experiments on two popular public datasets demonstrate that RAGFormer achieves state-of-the-art performance. The significant improvement of RAGFormer in an industrial credit card fraud detection dataset further validates the applicability of our method in real-world business scenarios.
Predicting and Explaining Hearing Aid Usage Using Encoder-Decoder with Attention Mechanism and SHAP
It is essential to understand the personal, behavioral, environmental, and other factors that correlate with optimal hearing aid fitting and hearing aid users' experiences in order to improve hearing loss patient satisfaction and quality of life, as well as reduce societal and financial burdens. This work proposes a novel framework that uses Encoder-decoder with attention mechanism (attn-ED) for predicting future hearing aid usage and SHAP to explain the factors contributing to this prediction. It has been demonstrated in experiments that attn-ED performs well at predicting future hearing aid usage, and that SHAP can be utilized to calculate the contribution of different factors affecting hearing aid usage. This framework aims to establish confidence that AI models can be utilized in the medical domain with the use of XAI methods. Moreover, the proposed framework can also assist clinicians in determining the nature of interventions.
Joint Analysis of Single-Cell Data across Cohorts with Missing Modalities
Arriola, Marianne, Pan, Weishen, Zhou, Manqi, Zhang, Qiannan, Su, Chang, Wang, Fei
Joint analysis of multi-omic single-cell data across cohorts has significantly enhanced the comprehensive analysis of cellular processes. However, most of the existing approaches for this purpose require access to samples with complete modality availability, which is impractical in many real-world scenarios. In this paper, we propose (Single-Cell Cross-Cohort Cross-Category) integration, a novel framework that learns unified cell representations under domain shift without requiring full-modality reference samples. Our generative approach learns rich cross-modal and cross-domain relationships that enable imputation of these missing modalities. Through experiments on real-world multi-omic datasets, we demonstrate that offers a robust solution to single-cell tasks such as cell type clustering, cell type classification, and feature imputation.
Excess Delay from GDP: Measurement and Causal Analysis
Ground Delay Programs (GDPs) have been widely used to resolve excessive demand-capacity imbalances at arrival airports by shifting foreseen airborne delay to pre-departure ground delay. While offering clear safety and efficiency benefits, GDPs may also create additional delay because of imperfect execution and uncertainty in predicting arrival airport capacity. This paper presents a methodology for measuring excess delay resulting from individual GDPs and investigates factors that influence excess delay using regularized regression models. We measured excess delay for 1210 GDPs from 33 U.S. airports in 2019. On a per-restricted flight basis, the mean excess delay is 35.4 min with std of 20.6 min. In our regression analysis of the variation in excess delay, ridge regression is found to perform best. The factors affecting excess delay include time variations during gate out and taxi out for flights subject to the GDP, program rate setting and revisions, and GDP time duration.
Motion Prediction with Gaussian Processes for Safe Human-Robot Interaction in Virtual Environments
Mugisha, Stanley, Guda, Vamsi Krishna, Chevallereau, Christine, Chablat, Damien, Zoppi, Matteo
Humans use collaborative robots as tools for accomplishing various tasks. The interaction between humans and robots happens in tight shared workspaces. However, these machines must be safe to operate alongside humans to minimize the risk of accidental collisions. Ensuring safety imposes many constraints, such as reduced torque and velocity limits during operation, thus increasing the time to accomplish many tasks. However, for applications such as using collaborative robots as haptic interfaces with intermittent contacts for virtual reality applications, speed limitations result in poor user experiences. This research aims to improve the efficiency of a collaborative robot while improving the safety of the human user. We used Gaussian process models to predict human hand motion and developed strategies for human intention detection based on hand motion and gaze to improve the time for the robot and human security in a virtual environment. We then studied the effect of prediction. Results from comparisons show that the prediction models improved the robot time by 3\% and safety by 17\%. When used alongside gaze, prediction with Gaussian process models resulted in an improvement of the robot time by 2\% and the safety by 13\%.
Adaptive Stabilization Based on Machine Learning for Column Generation
Shen, Yunzhuang, Sun, Yuan, Li, Xiaodong, Cao, Zhiguang, Eberhard, Andrew, Zhang, Guangquan
Column generation (CG) is a well-established method for solving large-scale linear programs. It involves iteratively optimizing a subproblem containing a subset of columns and using its dual solution to generate new columns with negative reduced costs. This process continues until the dual values converge to the optimal dual solution to the original problem. A natural phenomenon in CG is the heavy oscillation of the dual values during iterations, which can lead to a substantial slowdown in the convergence rate. Stabilization techniques are devised to accelerate the convergence of dual values by using information beyond the state of the current subproblem. However, there remains a significant gap in obtaining more accurate dual values at an earlier stage. To further narrow this gap, this paper introduces a novel approach consisting of 1) a machine learning approach for accurate prediction of optimal dual solutions and 2) an adaptive stabilization technique that effectively capitalizes on accurate predictions. On the graph coloring problem, we show that our method achieves a significantly improved convergence rate compared to traditional methods.
OTLP: Output Thresholding Using Mixed Integer Linear Programming
Koseoglu, Baran, Traverso, Luca, Topiwalla, Mohammed, Kraev, Egor, Szopory, Zoltan
Almost all classification methods such as XGBoost [1], Random Forest [2], Logistic Regression [3] are able to produce probability estimates. Output thresholding is a process to tune the decision threshold which is later used to assign class predictions based on a model's probability estimates for instances during inference [4]. For binary classification tasks, instances with probability estimates higher than or equal to the threshold are assigned positives class, otherwise as negative which is depicted in Table 1. Adjusting the threshold is particularly important for imbalanced classification problems where the train datasets have a smaller number of samples in the minority classes compared to the other classes. Output thresholding is one of the methods to address class imbalance problem [5]. Since the distribution of classes is skewed and probability estimates often favor the majority class, using a default classification threshold of 0.5 may not be the most effective approach for such problems [6]. Therefore it is essential to perform a search for the threshold to use during inference. Output thresholding is also considered to address class imbalance problem for convolutional neural networks [7].
Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines
Chang, Chaokun, Lo, Eric, Ye, Chunxiao
Machine learning inference pipelines commonly encountered in data science and industries often require real-time responsiveness due to their user-facing nature. However, meeting this requirement becomes particularly challenging when certain input features require aggregating a large volume of data online. Recent literature on interpretable machine learning reveals that most machine learning models exhibit a notable degree of resilience to variations in input. This suggests that machine learning models can effectively accommodate approximate input features with minimal discernible impact on accuracy. In this paper, we introduce Biathlon, a novel ML serving system that leverages the inherent resilience of models and determines the optimal degree of approximation for each aggregation feature. This approach enables maximum speedup while ensuring a guaranteed bound on accuracy loss. We evaluate Biathlon on real pipelines from both industry applications and data science competitions, demonstrating its ability to meet real-time latency requirements by achieving 5.3x to 16.6x speedup with almost no accuracy loss.
ST-Mamba: Spatial-Temporal Selective State Space Model for Traffic Flow Prediction
Shao, Zhiqi, Bell, Michael G. H., Wang, Ze, Geers, D. Glenn, Xi, Haoning, Gao, Junbin
Traffic flow prediction, a critical aspect of intelligent transportation systems, has been increasingly popular in the field of artificial intelligence, driven by the availability of extensive traffic data. The current challenges of traffic flow prediction lie in integrating diverse factors while balancing the trade-off between computational complexity and the precision necessary for effective long-range and large-scale predictions. To address these challenges, we introduce a Spatial-Temporal Selective State Space (ST-Mamba) model, which is the first to leverage the power of spatial-temporal learning in traffic flow prediction without using graph modeling. The ST-Mamba model can effectively capture the long-range dependency for traffic flow data, thereby avoiding the issue of over-smoothing. The proposed ST-Mamba model incorporates an effective Spatial-Temporal Mixer (ST-Mixer) to seamlessly integrate spatial and temporal data processing into a unified framework and employs a Spatial-Temporal Selective State Space (ST-SSM) block to improve computational efficiency. The proposed ST-Mamba model, specifically designed for spatial-temporal data, simplifies processing procedure and enhances generalization capabilities, thereby significantly improving the accuracy of long-range traffic flow prediction. Compared to the previous state-of-the-art (SOTA) model, the proposed ST-Mamba model achieves a 61.11\% improvement in computational speed and increases prediction accuracy by 0.67\%. Extensive experiments with real-world traffic datasets demonstrate that the \textsf{ST-Mamba} model sets a new benchmark in traffic flow prediction, achieving SOTA performance in computational efficiency for both long- and short-range predictions and significantly improving the overall efficiency and effectiveness of traffic management.
ReModels: Quantile Regression Averaging models
Zakrzewski, Grzegorz, Skonieczka, Kacper, Małkiński, Mikołaj, Mańdziuk, Jacek
Electricity price forecasts play a crucial role in making key business decisions within the electricity markets. A focal point in this domain are probabilistic predictions, which delineate future price values in a more comprehensive manner than simple point forecasts. The golden standard in probabilistic approaches to predict energy prices is the Quantile Regression Averaging (QRA) method. In this paper, we present a Python package that encompasses the implementation of QRA, along with modifications of this approach that have appeared in the literature over the past few years. The proposed package also facilitates the acquisition and preparation of data related to electricity markets, as well as the evaluation of model predictions.