Performance Analysis
Reviews: Gradients of Generative Models for Improved Discriminative Analysis of Tandem Mass Spectra
This paper introduces Theseus, an algorithm for matching MS/MS spectra to peptide in a D.B. This is a challenging and important task. It is important because MS/MS is currently practically the only common high-throughput method to identify which proteins are present in a sample. It is challenging because the data is analog (intensity vs. m/z graphs) and extremely noisy. This work builds upon an impressive body of work that has been dedicated to this problem.
Reviews: Estimating Accuracy from Unlabeled Data: A Probabilistic Logic Approach
This paper builds on the work of Platanios et al. (2014, 2016) on estimating the accuracy of a set of classifiers for a given task using only unlabeled data, based on the agreement behavior of the classifiers. The current work uses a probabilistic soft logic (PSL) model to infer the error rates of the classifiers. The paper also extends this approach to the case where we have multiple related classification tasks: for instance, classifying noun phrases with regard to their membership in multiple categories, some of which subsume others and some of which are mutually exclusive. The paper shows how a PSL model can take into account these constraints among the categories, yielding better error rate estimates and higher joint classification accuracy. It is well written and the methodology seems sound.
Reviews: Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples
In this paper the authors examine the intuition that interpretability to be the workhorse in detecting adversarial examples of different kinds. That is, if the humanly interpretable attributes are all the same for two images, then the prediction result should only be different if some non-interpretable neurons behave differently. Other than adversarial examples, this work is also highly related to interpretability and explainability questions for DNNs. The basis of their detection mechanism (AmI) lies in determining the sets of neurons (they call attribute witnesses) that are correspond (one-to-one) to a humanly interpretable attributes (like eyeglasses). That means, if the attribute does not change, the neuron should not give a different output, and the other way around if the feature changes, the neuron should change.
Predicting Fine-grained Behavioral and Psychological Symptoms of Dementia Based on Machine Learning and Smart Wearable Devices
Hsu, Benny Wei-Yun, Chen, Yu-Ming, Yang, Yuan-Han, Tseng, Vincent S.
Effective management and early detection of BPSD are crucial to reduce the stress and burden on caregivers and healthcare systems. Despite the advancements in machine learning for dementia prediction, there is a considerable gap in utilizing these methods for BPSD prediction. This study aims to fill this gap by presenting a novel personalized framework for BPSD prediction, utilizing physiological signals from smart wearable devices. Our personalized fine-grained BPSD prediction method accurately predicts BPSD occurrences by extracting individual behavioral patterns, while the generalized models identify diverse patterns and differentiate between various BPSD symptoms. Detailed comparisons between the proposed personalized method and conventional generalized methods reveals substantial improvements across all performance metrics, including a 16.0% increase in AUC. These results demonstrate the potential of our proposed method in advancing dementia care by enabling proactive interventions and improving patient outcomes in real-world scenarios. To the best of our knowledge, this is the first study that leverages physiological signals from smart wearable devices to predict BPSD, marking a significant stride in dementia care research.
Precision Cancer Classification and Biomarker Identification from mRNA Gene Expression via Dimensionality Reduction and Explainable AI
Tabassum, Farzana, Islam, Sabrina, Rizwan, Siana, Sobhan, Masrur, Ahmed, Tasnim, Ahmed, Sabbir, Chowdhury, Tareque Mohmud
Gene expression analysis is a critical method for cancer classification, enabling precise diagnoses through the identification of unique molecular signatures associated with various tumors. Identifying cancer-specific genes from gene expression values enables a more tailored and personalized treatment approach. However, the high dimensionality of mRNA gene expression data poses challenges for analysis and data extraction. This research presents a comprehensive pipeline designed to accurately identify 33 distinct cancer types and their corresponding gene sets. It incorporates a combination of normalization and feature selection techniques to reduce dataset dimensionality effectively while ensuring high performance. Notably, our pipeline successfully identifies a substantial number of cancer-specific genes using a reduced feature set of just 500, in contrast to using the full dataset comprising 19,238 features. By employing an ensemble approach that combines three top-performing classifiers, a classification accuracy of 96.61% was achieved. Furthermore, we leverage Explainable AI to elucidate the biological significance of the identified cancer-specific genes, employing Differential Gene Expression (DGE) analysis.
FAIREDU: A Multiple Regression-Based Method for Enhancing Fairness in Machine Learning Models for Educational Applications
Pham, Nga, Do, Minh Kha, Dai, Tran Vu, Hung, Pham Ngoc, Nguyen-Duc, Anh
Fairness in artificial intelligence and machine learning (AI/ML) models is becoming critically important, especially as decisions made by these systems impact diverse groups. In education, a vital sector for all countries, the widespread application of AI/ML systems raises specific concerns regarding fairness. Current research predominantly focuses on fairness for individual sensitive features, which limits the comprehensiveness of fairness assessments. This paper introduces FAIREDU, a novel and effective method designed to improve fairness across multiple sensitive features. Through extensive experiments, we evaluate FAIREDU effectiveness in enhancing fairness without compromising model performance. The results demonstrate that FAIREDU addresses intersectionality across features such as gender, race, age, and other sensitive features, outperforming state-of-the-art methods with minimal effect on model accuracy. The paper also explores potential future research directions to enhance further the method robustness and applicability to various machine-learning models and datasets.
Reliable Heading Tracking for Pedestrian Road Crossing Prediction Using Commodity Devices
Yang, Yucheng, Li, Jingjie, Fawaz, Kassem
Pedestrian heading tracking enables applications in pedestrian navigation, traffic safety, and accessibility. Previous works, using inertial sensor fusion or machine learning, are limited in that they assume the phone is fixed in specific orientations, hindering their generalizability. We propose a new heading tracking algorithm, the Orientation-Heading Alignment (OHA), which leverages a key insight: people tend to carry smartphones in certain ways due to habits, such as swinging them while walking. For each smartphone attitude during this motion, OHA maps the smartphone orientation to the pedestrian heading and learns such mappings efficiently from coarse headings and smartphone orientations. To anchor our algorithm in a practical scenario, we apply OHA to a challenging task: predicting when pedestrians are about to cross the road to improve road user safety. In particular, using 755 hours of walking data collected since 2020 from 60 individuals, we develop a lightweight model that operates in real-time on commodity devices to predict road crossings. Our evaluation shows that OHA achieves 3.4 times smaller heading errors across nine scenarios than existing methods. Furthermore, OHA enables the early and accurate detection of pedestrian crossing behavior, issuing crossing alerts 0.35 seconds, on average, before pedestrians enter the road range.
Beyond the Alphabet: Deep Signal Embedding for Enhanced DNA Clustering
Abraham, Hadas, Gahtan, Barak, Kobovich, Adir, Leitersdorf, Orian, Bronstein, Alex M., Yaakobi, Eitan
The rapid growth of digital data, projected to reach 180 zettabytes by 2025, is causing a data storage crisis that cannot be addressed by existing storage technologies [Rydning, 2022]. In response, deoxyribonucleic acid (DNA) is emerging as a promising alternative storage medium due to its incredible density and durability. The DNA storage process includes four stages illustrated in Figure 1: (1) an "encoding" stage in which binary data files are encoded into DNA strands (design files) using error-correcting code (ECC) [Koblitz et al., 2000] schemes that may also overcome errors, (2) a "synthesis" stage, which produces artificial DNA strands of each design strand and are then stored in a storage container [LeProust et al., 2010], (3) a "sequencing" stage [Anavy et al., 2019, Erlich and Zielinski, 2017, Organick et al., 2018, Yazdi et al., 2017] which translates a DNA strand into a digital sequence known as a "read", and (4) a "retrieval" stage where reads are decoded back to binary data files while correcting any errors using the chosen coding methods. Despite the vast potential of DNA storage, current DNA sequencers are yet to overcome challenges such as low throughput and high costs compared to the traditional alternatives [Alliance, 2021, Shomorony et al., 2022, Yazdi et al., 2015]. The emerging Nanopore technology offers real-time sequencing of DNA strands with drastically lower costs and portability compared to traditional Illumina sequencing machines [Jain et al., 2016, Kono and Arakawa, 2019]. Despite having higher error rates compared to other sequencing technologies such as Illumina, Nanopore sequencing is gaining significant attention due to its lower cost, portability, and capability to sequence longer strands of DNA.
Extreme Value Modelling of Feature Residuals for Anomaly Detection in Dynamic Graphs
Kandanaarachchi, Sevvandi, Sanderson, Conrad, Hyndman, Rob J.
Detecting anomalies in a temporal sequence of graphs can be applied is areas such as the detection of accidents in transport networks and cyber attacks in computer networks. Existing methods for detecting abnormal graphs can suffer from multiple limitations, such as high false positive rates as well as difficulties with handling variable-sized graphs and non-trivial temporal dynamics. To address this, we propose a technique where temporal dependencies are explicitly modelled via time series analysis of a large set of pertinent graph features, followed by using residuals to remove the dependencies. Extreme Value Theory is then used to robustly model and classify any remaining extremes, aiming to produce low false positives rates. Comparative evaluations on a multitude of graph instances show that the proposed approach obtains considerably better accuracy than TensorSplat and Laplacian Anomaly Detection.
Understanding with toy surrogate models in machine learning
Unlike regular models, these very simple models--often referred to as toy models--are not required to be linked to the real world through structural similarity or resemblance relations. They are not meant to be approximations of the target world system, and in some cases, they are not even required to be representational. In semantic terms, they do not accurately map onto their targets. Despite these limitations, they are still useful in understanding theoretical concepts and possible configurations of the target system. Paradigmatic examples of toy models include Boyle's law and the Ising model in physics, the Lotka-Volterra model in population ecology, and the Schelling model in the social sciences (Weisberg, 2013). In recent years, philosophers of science have become interested in toy models (Grüne-Yanoff, 2009; Luczak, 2017; Reutlinger et al., 2018; Frigg & Nguyen, 2017; Nguyen, 2020). The main purpose of this literature is to explore the nature of these models and examine how they perform their epistemic function. Despite lacking the regular descriptive and predictive features of full-scale scientific models, they often offer an elementary understanding of a phenomenon. Their definitions of "toy model" differ as well as their assessment of the importance of representation in modelling generally, but they all agree that toy models play an important epistemic role in scientific research, exploration, and pedagogy.