Statistical Learning
Regression under demographic parity constraints via unlabeled post-processing
Chzhen, Evgenii, Hebiri, Mohamed, Taturyan, Gayane
We address the problem of performing regression while ensuring demographic parity, even without access to sensitive attributes during inference. We present a general-purpose post-processing algorithm that, using accurate estimates of the regression function and a sensitive attribute predictor, generates predictions that meet the demographic parity constraint. Our method involves discretization and stochastic minimization of a smooth convex function. It is suitable for online post-processing and multi-class classification tasks only involving unlabeled data for the post-processing. Unlike prior methods, our approach is fully theory-driven. We require precise control over the gradient norm of the convex function, and thus, we rely on more advanced techniques than standard stochastic gradient descent. Our algorithm is backed by finite-sample analysis and post-processing bounds, with experimental results validating our theoretical findings.
Decoding Digital Influence: The Role of Social Media Behavior in Scientific Stratification Through Logistic Attribution Method
Scientific social stratification is a classic theme in the sociology of science. The deep integration of social media has bridged the gap between scientometrics and sociology of science. This study comprehensively analyzes the impact of social media on scientific stratification and mobility, delving into the complex interplay between academic status and social media activity in the digital age. [Research Method] Innovatively, this paper employs An Explainable Logistic Attribution Analysis from a meso-level perspective to explore the correlation between social media behaviors and scientific social stratification. It examines the impact of scientists' use of social media in the digital age on scientific stratification and mobility, uniquely combining statistical methods with machine learning. This fusion effectively integrates hypothesis testing with a substantive interpretation of the contribution of independent variables to the model. [Research Conclusion] Empirical evidence demonstrates that social media promotes stratification and mobility within the scientific community, revealing a nuanced and non-linear facilitation mechanism. Social media activities positively impact scientists' status within the scientific social hierarchy to a certain extent, but beyond a specific threshold, this impact turns negative. It shows that the advent of social media has opened new channels for academic influence, transcending the limitations of traditional academic publishing, and prompting changes in scientific stratification. Additionally, the study acknowledges the limitations of its experimental design and suggests future research directions.
A New Theoretical Perspective on Data Heterogeneity in Federated Optimization
Wang, Jiayi, Wang, Shiqiang, Chen, Rong-Rong, Ji, Mingyue
In federated learning (FL), data heterogeneity is the main reason that existing theoretical analyses are pessimistic about the convergence rate. In particular, for many FL algorithms, the convergence rate grows dramatically when the number of local updates becomes large, especially when the product of the gradient divergence and local Lipschitz constant is large. However, empirical studies can show that more local updates can improve the convergence rate even when these two parameters are large, which is inconsistent with the theoretical findings. This paper aims to bridge this gap between theoretical understanding and practical performance by providing a theoretical analysis from a new perspective on data heterogeneity. In particular, we propose a new and weaker assumption compared to the local Lipschitz gradient assumption, named the heterogeneity-driven pseudo-Lipschitz assumption. We show that this and the gradient divergence assumptions can jointly characterize the effect of data heterogeneity. By deriving a convergence upper bound for FedAvg and its extensions, we show that, compared to the existing works, local Lipschitz constant is replaced by the much smaller heterogeneity-driven pseudo-Lipschitz constant and the corresponding convergence upper bound can be significantly reduced for the same number of local updates, although its order stays the same. In addition, when the local objective function is quadratic, more insights on the impact of data heterogeneity can be obtained using the heterogeneity-driven pseudo-Lipschitz constant. For example, we can identify a region where FedAvg can outperform mini-batch SGD even when the gradient divergence can be arbitrarily large. Our findings are validated using experiments.
Offline Imitation Learning Through Graph Search and Retrieval
Yin, Zhao-Heng, Abbeel, Pieter
Imitation learning is a powerful machine learning algorithm for a robot to acquire manipulation skills. Nevertheless, many real-world manipulation tasks involve precise and dexterous robot-object interactions, which make it difficult for humans to collect high-quality expert demonstrations. As a result, a robot has to learn skills from suboptimal demonstrations and unstructured interactions, which remains a key challenge. Existing works typically use offline deep reinforcement learning (RL) to solve this challenge, but in practice these algorithms are unstable and fragile due to the deadly triad issue. To overcome this problem, we propose GSR, a simple yet effective algorithm that learns from suboptimal demonstrations through Graph Search and Retrieval. We first use pretrained representation to organize the interaction experience into a graph and perform a graph search to calculate the values of different behaviors. Then, we apply a retrieval-based procedure to identify the best behavior (actions) on each state and use behavior cloning to learn that behavior. We evaluate our method in both simulation and real-world robotic manipulation tasks with complex visual inputs, covering various precise and dexterous manipulation skills with objects of different physical properties. GSR can achieve a 10% to 30% higher success rate and over 30% higher proficiency compared to baselines. Our project page is at https://zhaohengyin.github.io/gsr.
GFE-Mamba: Mamba-based AD Multi-modal Progression Assessment via Generative Feature Extraction from MCI
Fang, Zhaojie, Zhu, Shenghao, Chen, Yifei, Zou, Binfeng, Jia, Fan, Qiu, Linwei, Liu, Chang, Huang, Yiyu, Feng, Xiang, Qin, Feiwei, Wang, Changmiao, Wang, Yeru, Fan, Jin, Chu, Changbiao, Wu, Wan-Zhen, Zhao, Hu
--Alzheimer's Disease (AD) is a progressive, irreversible neurodegenerative disorder that often originates from Mild Cognitive Impairment (MCI). This progression results in significant memory loss and severely affects patients' quality of life. Clinical trials have consistently shown that early and targeted interventions for individuals with MCI may slow or even prevent the advancement of AD. Research indicates that accurate medical classification requires diverse multimodal data, including detailed assessment scales and neuroimaging techniques like Magnetic Resonance Imaging (MRI) and Positron Emission T omography (PET). However, simultaneously collecting the aforementioned three modalities for training presents substantial challenges. T o tackle these difficulties, we propose GFE-Mamba, a multimodal classifier founded on Generative Feature Extractor . The intermediate features provided by this Extractor can compensate for the shortcomings of PET and achieve profound multimodal fusion in the classifier . The Mamba block, as the backbone of the classifier, enables it to efficiently extract information from long-sequence scale information. We provide our rationale for developing this cross-temporal progression prediction dataset and the pre-trained Extractor weights. Our experimental findings reveal that the GFE-Mamba model effectively predicts the progression from MCI to AD and surpasses several leading methods in the field. Data used in the preparation of this article was obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). The ADNI researchers contributed data but did not participate in analysis or writing of this report. The data was made available at the ADNI database (www.loni.usc.edu). Changbiao Chu is with Xuanwu Hospital, Capital Medical University, Beijing, 100053, China (e-mail:chucb@xwhosp.org). LZHEIMER'S Disease (AD) is a prevalent neurode-generative condition in older adults, affecting memory, cognitive abilities, and daily life activities [41].
Explainable AI-based Intrusion Detection System for Industry 5.0: An Overview of the Literature, associated Challenges, the existing Solutions, and Potential Research Directions
Khan, Naseem, Ahmad, Kashif, Tamimi, Aref Al, Alani, Mohammed M., Bermak, Amine, Khalil, Issa
Industry 5.0, which focuses on human and Artificial Intelligence (AI) collaboration for performing different tasks in manufacturing, involves a higher number of robots, Internet of Things (IoTs) devices and interconnections, Augmented/Virtual Reality (AR), and other smart devices. The huge involvement of these devices and interconnection in various critical areas, such as economy, health, education and defense systems, poses several types of potential security flaws. AI itself has been proven a very effective and powerful tool in different areas of cybersecurity, such as intrusion detection, malware detection, and phishing detection, among others. Just as in many application areas, cybersecurity professionals were reluctant to accept black-box ML solutions for cybersecurity applications. This reluctance pushed forward the adoption of eXplainable Artificial Intelligence (XAI) as a tool that helps explain how decisions are made in ML-based systems. In this survey, we present a comprehensive study of different XAI-based intrusion detection systems for industry 5.0, and we also examine the impact of explainability and interpretability on Cybersecurity practices through the lens of Adversarial XIDS (Adv-XIDS) approaches. Furthermore, we analyze the possible opportunities and challenges in XAI cybersecurity systems for industry 5.0 that elicit future research toward XAI-based solutions to be adopted by high-stakes industry 5.0 applications. We believe this rigorous analysis will establish a foundational framework for subsequent research endeavors within the specified domain.
Deep multimodal saliency parcellation of cerebellar pathways: linking microstructure and individual function through explainable multitask learning
Tchetchenian, Ari, Zekelman, Leo, Chen, Yuqian, Rushmore, Jarrett, Zhang, Fan, Yeterian, Edward H., Makris, Nikos, Rathi, Yogesh, Meijering, Erik, Song, Yang, O'Donnell, Lauren J.
Parcellation of human cerebellar pathways is essential for advancing our understanding of the human brain. Existing diffusion MRI tractography parcellation methods have been successful in defining major cerebellar fibre tracts, while relying solely on fibre tract structure. However, each fibre tract may relay information related to multiple cognitive and motor functions of the cerebellum. Hence, it may be beneficial for parcellation to consider the potential importance of the fibre tracts for individual motor and cognitive functional performance measures. In this work, we propose a multimodal data-driven method for cerebellar pathway parcellation, which incorporates both measures of microstructure and connectivity, and measures of individual functional performance. Our method involves first training a multitask deep network to predict various cognitive and motor measures from a set of fibre tract structural features. The importance of each structural feature for predicting each functional measure is then computed, resulting in a set of structure-function saliency values that are clustered to parcellate cerebellar pathways. We refer to our method as Deep Multimodal Saliency Parcellation (DeepMSP), as it computes the saliency of structural measures for predicting cognitive and motor functional performance, with these saliencies being applied to the task of parcellation. Applying DeepMSP we found that it was feasible to identify multiple cerebellar pathway parcels with unique structure-function saliency patterns that were stable across training folds.
Privacy-Preserving Multi-Center Differential Protein Abundance Analysis with FedProt
Burankova, Yuliya, Abele, Miriam, Bakhtiari, Mohammad, von Törne, Christine, Barth, Teresa, Schweizer, Lisa, Giesbertz, Pieter, Schmidt, Johannes R., Kalkhof, Stefan, Müller-Deile, Janina, van Veelen, Peter A, Mohammed, Yassene, Hammer, Elke, Arend, Lis, Adamowicz, Klaudia, Laske, Tanja, Hartebrodt, Anne, Frisch, Tobias, Meng, Chen, Matschinske, Julian, Späth, Julian, Röttger, Richard, Schwämmle, Veit, Hauck, Stefanie M., Lichtenthaler, Stefan, Imhof, Axel, Mann, Matthias, Ludwig, Christina, Kuster, Bernhard, Baumbach, Jan, Zolotareva, Olga
Quantitative mass spectrometry has revolutionized proteomics by enabling simultaneous quantification of thousands of proteins. Pooling patient-derived data from multiple institutions enhances statistical power but raises significant privacy concerns. Here we introduce FedProt, the first privacy-preserving tool for collaborative differential protein abundance analysis of distributed data, which utilizes federated learning and additive secret sharing. In the absence of a multicenter patient-derived dataset for evaluation, we created two, one at five centers from LFQ E.coli experiments and one at three centers from TMT human serum. Evaluations using these datasets confirm that FedProt achieves accuracy equivalent to DEqMS applied to pooled data, with completely negligible absolute differences no greater than $\text{$4 \times 10^{-12}$}$. In contrast, -log10(p-values) computed by the most accurate meta-analysis methods diverged from the centralized analysis results by up to 25-27. FedProt is available as a web tool with detailed documentation as a FeatureCloud App.
${\it Asparagus}$: A Toolkit for Autonomous, User-Guided Construction of Machine-Learned Potential Energy Surfaces
Töpfer, Kai, Vazquez-Salazar, Luis Itza, Meuwly, Markus
With the establishment of machine learning (ML) techniques in the scientific community, the construction of ML potential energy surfaces (ML-PES) has become a standard process in physics and chemistry. So far, improvements in the construction of ML-PES models have been conducted independently, creating an initial hurdle for new users to overcome and complicating the reproducibility of results. Aiming to reduce the bar for the extensive use of ML-PES, we introduce ${\it Asparagus}$, a software package encompassing the different parts into one coherent implementation that allows an autonomous, user-guided construction of ML-PES models. ${\it Asparagus}$ combines capabilities of initial data sampling with interfaces to ${\it ab initio}$ calculation programs, ML model training, as well as model evaluation and its application within other codes such as ASE or CHARMM. The functionalities of the code are illustrated in different examples, including the dynamics of small molecules, the representation of reactive potentials in organometallic compounds, and atom diffusion on periodic surface structures. The modular framework of ${\it Asparagus}$ is designed to allow simple implementations of further ML-related methods and models to provide constant user-friendly access to state-of-the-art ML techniques.
Conformal Predictions under Markovian Data
Zheng, Frédéric, Proutiere, Alexandre
We study the split Conformal Prediction method when applied to Markovian data. We quantify the gap in terms of coverage induced by the correlations in the data (compared to exchangeable data). This gap strongly depends on the mixing properties of the underlying Markov chain, and we prove that it typically scales as $\sqrt{t_\mathrm{mix}\ln(n)/n}$ (where $t_\mathrm{mix}$ is the mixing time of the chain). We also derive upper bounds on the impact of the correlations on the size of the prediction set. Finally we present $K$-split CP, a method that consists in thinning the calibration dataset and that adapts to the mixing properties of the chain. Its coverage gap is reduced to $t_\mathrm{mix}/(n\ln(n))$ without really affecting the size of the prediction set. We finally test our algorithms on synthetic and real-world datasets.