Goto

Collaborating Authors

 Statistical Learning


Communication-Efficient l_0 Penalized Least Square

arXiv.org Artificial Intelligence

In this paper, we propose a communication-efficient penalized regression algorithm for high-dimensional sparse linear regression models with massive data. This approach incorporates an optimized distributed system communication algorithm, named CESDAR algorithm, based on the Enhanced Support Detection and Root finding algorithm. The CESDAR algorithm leverages data distributed across multiple machines to compute and update the active set and introduces the communication-efficient surrogate likelihood framework to approximate the optimal solution for the full sample on the active set, resulting in the avoidance of raw data transmission, which enhances privacy and data security, while significantly improving algorithm execution speed and substantially reducing communication costs. Notably, this approach achieves the same statistical accuracy as the global estimator. Furthermore, this paper explores an extended version of CESDAR and an adaptive version of CESDAR to enhance algorithmic speed and optimize parameter selection, respectively. Simulations and real data benchmarks experiments demonstrate the efficiency and accuracy of the CESDAR algorithm. Introduction The rapid development of data collection techniques has led to unprecedented growth and expansion in both the volume and dimensionality of data. The massive high-dimensional datasets entail high computational costs and memory constraints. Numerous methods have been utilized for variable selection and parameter estimation in the research domain, including LASSO [1], adaptive LASSO [2], the smoothly clipped absolute deviation (SCAD) penalty [3], the minimax concave penalty (MCP) [4] and so on.


Example-Based Concept Analysis Framework for Deep Weather Forecast Models

arXiv.org Artificial Intelligence

To improve the trustworthiness of an AI model, finding consistent, understandable representations of its inference process is essential. This understanding is particularly important in high-stakes operations such as weather forecasting, where the identification of underlying meteorological mechanisms is as critical as the accuracy of the predictions. Despite the growing literature that addresses this issue through explainable AI, the applicability of their solutions is often limited due to their AI-centric development. To fill this gap, we follow a user-centric process to develop an example-based concept analysis framework, which identifies cases that follow a similar inference process as the target instance in a target model and presents them in a user-comprehensible format. Our framework provides the users with visually and conceptually analogous examples, including the probability of concept assignment to resolve ambiguities in weather mechanisms. To bridge the gap between vector representations identified from models and human-understandable explanations, we compile a human-annotated concept dataset and implement a user interface to assist domain experts involved in the the framework development.


ffstruc2vec: Flat, Flexible and Scalable Learning of Node Representations from Structural Identities

arXiv.org Artificial Intelligence

These embeddings can be leveraged in various downstream tasks, including node classification, link prediction, clustering, exploratory data analysis, and network visualization. The method has found broad application across diverse domains, such as fraud detection in financial networks (van Belle et al. 2023), friendship recommendation and bot detection in social networks (Saxena et al. 2022; Dehghan et al. 2023), knowledge discovery in knowledge graphs (Egami et al. 2023; Liu et al. 2023), analysis of biological networks (Jiang et al. 2021; Pasquier et al. 2023), and fake review detection on online platforms (Zaki et al. 2024). A key challenge in Node Embedding is developing a scalable method for preserving the structural properties of nodes suitable for the required structural patterns of a downstream application task. The type of structural patterns in which a node is embedded within the graph can vary depending on the role or function of the node in a specific application task. For instance, fraudulent activities such as money laundering can be embedded in particular money flow patterns among illicit entities, resulting in characteristic structural patterns within the financial transaction network, such as suspicious cyclic transaction chains (Granados Vargas 2022). These structural patterns differ significantly from those observed in social networks, where specific roles such as bridge and core nodes define the network's connectivity and influence (Huang et al. 2014). As Node Embedding methods cannot preserve all types of structural patterns simultaneously, they must align with the requirements of a specific application task when defining types of structural identities.


Benchmarking Federated Machine Unlearning methods for Tabular Data

arXiv.org Artificial Intelligence

Machine unlearning, which enables a model to forget specific data upon request, is increasingly relevant in the era of privacy-centric machine learning, particularly within federated learning (FL) environments. This paper presents a pioneering study on benchmarking machine unlearning methods within a federated setting for tabular data, addressing the unique challenges posed by cross-silo FL where data privacy and communication efficiency are paramount. We explore unlearning at the feature and instance levels, employing both machine learning, random forest and logistic regression models. Our methodology benchmarks various unlearning algorithms, including fine-tuning and gradient-based approaches, across multiple datasets, with metrics focused on fidelity, certifiability, and computational efficiency. Experiments demonstrate that while fidelity remains high across methods, tree-based models excel in certifiability, ensuring exact unlearning, whereas gradient-based methods show improved computational efficiency. This study provides critical insights into the design and selection of unlearning algorithms tailored to the FL environment, offering a foundation for further research in privacy-preserving machine learning.


Explainable AI-Based Interface System for Weather Forecasting Model

arXiv.org Artificial Intelligence

Machine learning (ML) is becoming increasingly popular in meteorological decision-making. Although the literature on explainable artificial intelligence (XAI) is growing steadily, user-centered XAI studies have not extend to this domain yet. This study defines three requirements for explanations of black-box models in meteorology through user studies: statistical model performance for different rainfall scenarios to identify model bias, model reasoning, and the confidence of model outputs. Appropriate XAI methods are mapped to each requirement, and the generated explanations are tested quantitatively and qualitatively. An XAI interface system is designed based on user feedback. The results indicate that the explanations increase decision utility and user trust. Users prefer intuitive explanations over those based on XAI algorithms even for potentially easy-to-recognize examples. These findings can provide evidence for future research on user-centered XAI algorithms, as well as a basis to improve the usability of AI systems in practice.


A stochastic gradient descent algorithm with random search directions

arXiv.org Machine Learning

Firstly, we introduce some notations. Secondly, we shall formulate our new class of SGD algorithms with random search directions which includes the SCGD algorithm. Finally, we will spell out some regularity assumptions. The SCGD algorithm as given in (3), represents the practical definition by considering the vectors in the canonical basis of R d . However, we can extend this coordinate selecting rule by using more general random vectors with a possible adaptive sampling policy. Therefore, we introduce the Stochastic Coordinate Gradient Descent algorithm with Random Search Direction (SCORS) defined for all n 1, by X n +1= X n γ nD (V n +1) f U n+1( X n), (SCORS) where the initial state X 1 is a squared integrable random vector of R d which can be arbitrarily chosen, D (v) = vv T for any vector v R d, the sequence (U n) is independent from (X n) where ( U n) is independent and identically distributed with U ([ [1, N ] ])distribution and V n is a random vector of R d sampled from an underlying distribution P n satisfying certain conditions (see Assumption 1 below). Furthermore, we 3 assume that V n +1is independent from U n +1conditionally on F n, where F n = σ (X 1, . . .


Explainable post-training bias mitigation with distribution-based fairness metrics

arXiv.org Artificial Intelligence

Machine learning (ML) techniques have become ubiquitous in the financial industry due to their powerful predictive performance. However, ML model outputs may lead to certain types of unintended bias, which are measures of unfairness that impact protected sub-populations. Predictive models, and strategies that rely on such models, are subject to laws and regulations that ensure fairness. For instance, financial institutions (FIs) in the U.S. that are in the business of extending credit to applicants are subject to the Equal Credit Opportunity Act (ECOA) [14] and the Fair Housing Act (FHA) [13], which prohibit discrimination in credit offerings and housing transactions. The protected classes identified in the laws, including race, gender, age (subject to very limited exceptions), ethnicity, national origin, and material status, cannot be used as attributes in lending decisions.


Suite-IN++: A FlexiWear BodyNet Integrating Global and Local Motion Features from Apple Suite for Robust Inertial Navigation

arXiv.org Artificial Intelligence

The proliferation of wearable technology has established multi-device ecosystems comprising smartphones, smartwatches, and headphones as critical enablers for ubiquitous pedestrian localization. However, traditional pedestrian dead reckoning (PDR) struggles with diverse motion modes, while data-driven methods, despite improving accuracy, often lack robustness due to their reliance on a single-device setup. Therefore, a promising solution is to fully leverage existing wearable devices to form a flexiwear bodynet for robust and accurate pedestrian localization. This paper presents Suite-IN++, a deep learning framework for flexiwear bodynet-based pedestrian localization. Suite-IN++ integrates motion data from wearable devices on different body parts, using contrastive learning to separate global and local motion features. It fuses global features based on the data reliability of each device to capture overall motion trends and employs an attention mechanism to uncover cross-device correlations in local features, extracting motion details helpful for accurate localization. To evaluate our method, we construct a real-life flexiwear bodynet dataset, incorporating Apple Suite (iPhone, Apple Watch, and AirPods) across diverse walking modes and device configurations. Experimental results demonstrate that Suite-IN++ achieves superior localization accuracy and robustness, significantly outperforming state-of-the-art models in real-life pedestrian tracking scenarios.


GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environments

arXiv.org Artificial Intelligence

The era of foundation models has revolutionized AI research, yet Graph Foundation Models (GFMs) remain constrained by the scarcity of large-scale graph corpora. Traditional graph data synthesis techniques primarily focus on simplistic structural operations, lacking the capacity to generate semantically rich nodes with meaningful textual attributes: a critical limitation for real-world applications. While large language models (LLMs) demonstrate exceptional text generation capabilities, their direct application to graph synthesis is impeded by context window limitations, hallucination phenomena, and structural consistency challenges. To address these issues, we introduce GraphMaster, the first multi-agent framework specifically designed for graph data synthesis in data-limited environments. GraphMaster orchestrates four specialized LLM agents (Manager, Perception, Enhancement, and Evaluation) that collaboratively optimize the synthesis process through iterative refinement, ensuring both semantic coherence and structural integrity. To rigorously evaluate our approach, we create new data-limited "Sub" variants of six standard graph benchmarks, specifically designed to test synthesis capabilities under realistic constraints. Additionally, we develop a novel interpretability assessment framework that combines human evaluation with a principled Grassmannian manifold-based analysis, providing both qualitative and quantitative measures of semantic coherence. Experimental results demonstrate that GraphMaster significantly outperforms traditional synthesis methods across multiple datasets, establishing a strong foundation for advancing GFMs in data-scarce environments.


Sim-is-More: Randomizing HW-NAS with Synthetic Devices

arXiv.org Artificial Intelligence

Existing hardware-aware NAS (HW-NAS) methods typically assume access to precise information circa the target device, either via analytical approximations of the post-compilation latency model, or through learned latency predictors. Such approximate approaches risk introducing estimation errors that may prove detrimental in risk-sensitive applications. In this work, we propose a two-stage HW-NAS framework, in which we first learn an architecture controller on a distribution of synthetic devices, and then directly deploy the controller on a target device. At test-time, our network controller deploys directly to the target device without relying on any pre-collected information, and only exploits direct interactions. In particular, the pre-training phase on synthetic devices enables the controller to design an architecture for the target device by interacting with it through a small number of high-fidelity latency measurements. To guarantee accessibility of our method, we only train our controller with training-free accuracy proxies, allowing us to scale the meta-training phase without incurring the overhead of full network training. We benchmark on HW-NATS-Bench, demonstrating that our method generalizes to unseen devices and searches for latency-efficient architectures by in-context adaptation using only a few real-world latency evaluations at test-time.