Statistical Learning
Guarding Digital Privacy: Exploring User Profiling and Security Enhancements
Kohli, Rishika, Gupta, Shaifu, Gaur, Manoj Singh
User profiling, the practice of collecting user information for personalized recommendations, has become widespread, driving progress in technology. However, this growth poses a threat to user privacy, as devices often collect sensitive data without their owners' awareness. This article aims to consolidate knowledge on user profiling, exploring various approaches and associated challenges. Through the lens of two companies sharing user data and an analysis of 18 popular Android applications in India across various categories, including $\textit{Social, Education, Entertainment, Travel, Shopping and Others}$, the article unveils privacy vulnerabilities. Further, the article propose an enhanced machine learning framework, employing decision trees and neural networks, that improves state-of-the-art classifiers in detecting personal information exposure. Leveraging the XAI (explainable artificial intelligence) algorithm LIME (Local Interpretable Model-agnostic Explanations), it enhances interpretability, crucial for reliably identifying sensitive data. Results demonstrate a noteworthy performance boost, achieving a $75.01\%$ accuracy with a reduced training time of $3.62$ seconds for neural networks. Concluding, the paper suggests research directions to strengthen digital security measures.
AiGAS-dEVL-RC: An Adaptive Growing Neural Gas Model for Recurrently Drifting Unsupervised Data Streams
Arostegi, Maria, Bilbao, Miren Nekane, Lobo, Jesus L., Del Ser, Javier
--Concept drift and extreme verification latency pose significant challenges in data stream learning, particularly when dealing with recurring concept changes in dynamic environments. This work introduces a novel method based on the Growing Neural Gas (GNG) algorithm, designed to effectively handle abrupt recurrent drifts while adapting to incrementally evolving data distributions (incremental drifts). Leveraging the self-organizing and topological adaptability of GNG, the proposed approach maintains a compact yet informative memory structure, allowing it to efficiently store and retrieve knowledge of past or recurring concepts, even under conditions of delayed or sparse stream supervision. Our experiments highlight the superiority of our approach over existing data stream learning methods designed to cope with incremental non-stationarities and verification latency, demonstrating its ability to quickly adapt to new drifts, robustly manage recurring patterns, and maintain high predictive accuracy with a minimal memory footprint. Unlike other techniques that fail to leverage recurring knowledge, our proposed approach is proven to be a robust and efficient online learning solution for unsupervised drifting data flows. Data stream learning has become increasingly relevant in a variety of real-world applications, ranging from fraud detection and stock market analysis to personalized recommendations and industrial process monitoring [1]. These systems rely on continuous real-time processing of data streams to make predictions or decisions. Unlike static datasets, data streams are often characterized by their unbounded, high-speed nature, which necessitates models that can operate incrementally, efficiently, and with minimal reliance on labeled data. Ensuring that such models remain accurate and adaptive over time is crucial for maintaining the performance of systems operating in dynamic environments [2]-[4].
Hyperbolic Diffusion Recommender Model
Yuan, Meng, Xiao, Yutian, Chen, Wei, Zhao, Chu, Wang, Deqing, Zhuang, Fuzhen
Diffusion models (DMs) have emerged as the new state-of-the-art family of deep generative models. To gain deeper insights into the limitations of diffusion models in recommender systems, we investigate the fundamental structural disparities between images and items. Consequently, items often exhibit distinct anisotropic and directional structures that are less prevalent in images. However, the traditional forward diffusion process continuously adds isotropic Gaussian noise, causing anisotropic signals to degrade into noise, which impairs the semantically meaningful representations in recommender systems. Inspired by the advancements in hyperbolic spaces, we propose a novel \textit{\textbf{H}yperbolic} \textit{\textbf{D}iffusion} \textit{\textbf{R}ecommender} \textit{\textbf{M}odel} (named HDRM). Unlike existing directional diffusion methods based on Euclidean space, the intrinsic non-Euclidean structure of hyperbolic space makes it particularly well-adapted for handling anisotropic diffusion processes. In particular, we begin by formulating concepts to characterize latent directed diffusion processes within a geometrically grounded hyperbolic space. Subsequently, we propose a novel hyperbolic latent diffusion process specifically tailored for users and items. Drawing upon the natural geometric attributes of hyperbolic spaces, we impose structural restrictions on the space to enhance hyperbolic diffusion propagation, thereby ensuring the preservation of the intrinsic topology of user-item graphs. Extensive experiments on three benchmark datasets demonstrate the effectiveness of HDRM.
Adaptive Shrinkage Estimation For Personalized Deep Kernel Regression In Modeling Brain Trajectories
Tassopoulou, Vasiliki, Shou, Haochang, Davatzikos, Christos
Longitudinal biomedical studies monitor individuals over time to capture dynamics in brain development, disease progression, and treatment effects. However, estimating trajectories of brain biomarkers is challenging due to biological variability, inconsistencies in measurement protocols (e.g., differences in MRI scanners), scarcity, and irregularity in longitudinal measurements. Herein, we introduce a novel personalized deep kernel regression framework for forecasting brain biomarkers, with application to regional volumetric measurements. Our approach integrates two key components: a population model that captures brain trajectories from a large and diverse cohort, and a subject-specific model that captures individual trajectories. To optimally combine these, we propose Adaptive Shrinkage Estimation, which effectively balances population and subject-specific models. We assess our model's performance through predictive accuracy metrics, uncertainty quantification, and validation against external clinical studies. Benchmarking against state-of-the-art statistical and machine learning models -- including linear mixed effects models, generalized additive models, and deep learning methods -- demonstrates the superior predictive performance of our approach. Additionally, we apply our method to predict trajectories of composite neuroimaging biomarkers, which highlights the versatility of our approach in modeling the progression of longitudinal neuroimaging biomarkers. Furthermore, validation on three external neuroimaging studies confirms the robustness of our method across different clinical contexts. We make the code available at https://github.com/vatass/AdaptiveShrinkageDKGP.
Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows
Rux, Nicolaj, Quellmalz, Michael, Steidl, Gabriele
Negative distance kernels $K(x,y) := - \|x-y\|$ were used in the definition of maximum mean discrepancies (MMDs) in statistics and lead to favorable numerical results in various applications. In particular, so-called slicing techniques for handling high-dimensional kernel summations profit from the simple parameter-free structure of the distance kernel. However, due to its non-smoothness in $x=y$, most of the classical theoretical results, e.g. on Wasserstein gradient flows of the corresponding MMD functional do not longer hold true. In this paper, we propose a new kernel which keeps the favorable properties of the negative distance kernel as being conditionally positive definite of order one with a nearly linear increase towards infinity and a simple slicing structure, but is Lipschitz differentiable now. Our construction is based on a simple 1D smoothing procedure of the absolute value function followed by a Riemann-Liouville fractional integral transform. Numerical results demonstrate that the new kernel performs similarly well as the negative distance kernel in gradient descent methods, but now with theoretical guarantees.
Deep Distributional Learning with Non-crossing Quantile Network
Shen, Guohao, Dai, Runpeng, Wu, Guojun, Luo, Shikai, Shi, Chengchun, Zhu, Hongtu
In this paper, we introduce a non-crossing quantile (NQ) network for conditional distribution learning. By leveraging non-negative activation functions, the NQ network ensures that the learned distributions remain monotonic, effectively addressing the issue of quantile crossing. Furthermore, the NQ network-based deep distributional learning framework is highly adaptable, applicable to a wide range of applications, from classical non-parametric quantile regression to more advanced tasks such as causal effect estimation and distributional reinforcement learning (RL). We also develop a comprehensive theoretical foundation for the deep NQ estimator and its application to distributional RL, providing an in-depth analysis that demonstrates its effectiveness across these domains. Our experimental results further highlight the robustness and versatility of the NQ network.
Gradient-based Sample Selection for Faster Bayesian Optimization
Wei, Qiyu, Wang, Haowei, Cao, Zirui, Wang, Songhao, Allmendinger, Richard, Álvarez, Mauricio A
Bayesian optimization (BO) is an effective technique for black-box optimization. However, its applicability is typically limited to moderate-budget problems due to the cubic complexity in computing the Gaussian process (GP) surrogate model. In large-budget scenarios, directly employing the standard GP model faces significant challenges in computational time and resource requirements. In this paper, we propose a novel approach, gradient-based sample selection Bayesian Optimization (GSSBO), to enhance the computational efficiency of BO. The GP model is constructed on a selected set of samples instead of the whole dataset. These samples are selected by leveraging gradient information to maintain diversity and representation. We provide a theoretical analysis of the gradient-based sample selection strategy and obtain explicit sublinear regret bounds for our proposed framework. Extensive experiments on synthetic and real-world tasks demonstrate that our approach significantly reduces the computational cost of GP fitting in BO while maintaining optimization performance comparable to baseline methods.
Double Machine Learning for Causal Inference under Shared-State Interference
Researchers and practitioners often wish to measure treatment effects in settings where units interact via markets and recommendation systems. In these settings, units are affected by certain shared states, like prices, algorithmic recommendations or social signals. We formalize this structure, calling it shared-state interference, and argue that our formulation captures many relevant applied settings. Our key modeling assumption is that individuals' potential outcomes are independent conditional on the shared state. We then prove an extension of a double machine learning (DML) theorem providing conditions for achieving efficient inference under shared-state interference. We also instantiate our general theorem in several models of interest where it is possible to efficiently estimate the average direct effect (ADE) or global average treatment effect (GATE).
FACT: Multinomial Misalignment Classification for Point Cloud Registration
Dillén, Ludvig, Forssén, Per-Erik, Edstedt, Johan
We present FACT, a method for predicting alignment quality (i.e., registration error) of registered lidar point cloud pairs. This is useful e.g. for quality assurance of large, automatically registered 3D models. FACT extracts local features from a registered pair and processes them with a point transformer-based network to predict a misalignment class. We generalize prior work that study binary alignment classification of registration errors, by recasting it as multinomial misalignment classification. To achieve this, we introduce a custom regression-by-classification loss function that combines the cross-entropy and Wasserstein losses, and demonstrate that it outperforms both direct regression and prior binary classification. FACT successfully classifies point-cloud pairs registered with both the classical ICP and GeoTransformer, while other choices, such as standard point-cloud-quality metrics and registration residuals are shown to be poor choices for predicting misalignment. On a synthetically perturbed point-cloud task introduced by the CorAl method, we show that FACT achieves substantially better performance than CorAl. Finally, we demonstrate how FACT can assist experts in correcting misaligned point-cloud maps. Our code is available at https://github.com/LudvigDillen/FACT_for_PCMC.
ASRL:A robust loss function with potential for development
Hui, Chenyu, Zhang, Anran, Li, Xintong
Abstract--In this article, we proposed a partition-wise robust loss function (ASRL -Adapative segmented robust loss)based on the previous robust loss function. The characteristics of this loss function are that it achieves high robustness and a wide range of applicability through partition-wise design and adaptive parameter adjustment. Finally, the advantages and development potential of this loss function were verified by applying this loss function to the XGBoost and using five different datasets (with different dimensions, different sample numbers, and different fields) to compare with the XGBoost using other loss functions. The results of multiple experiments have proven the advantages of ASRL in MSE, MAE, R2, etc. ASRL's dynamic segmentation design and adaptive threshold make it more robust and can be applied to more fields, such as as a loss function for multimodal learning and reinforcement learning, and has a large room for development.The implementation code repository github link in this paper is:ASRLCODE Index Terms--ASRL,Robustness,MSE,MAE,Loss Function I. INTRODUCTION In regression prediction of machine learning, the loss function is the core tool to measure the difference between the model prediction value and the true value. Its role runs through the entire process of model training, optimization and evaluation.