Performance Analysis
Multi-Task Learning for Post-transplant Cause of Death Analysis: A Case Study on Liver Transplant
Ding, Sirui, Tan, Qiaoyu, Chang, Chia-yuan, Zou, Na, Zhang, Kai, Hoot, Nathan R., Jiang, Xiaoqian, Hu, Xia
Organ transplant is the essential treatment method for some end-stage diseases, such as liver failure. Analyzing the post-transplant cause of death (CoD) after organ transplant provides a powerful tool for clinical decision making, including personalized treatment and organ allocation. However, traditional methods like Model for End-stage Liver Disease (MELD) score and conventional machine learning (ML) methods are limited in CoD analysis due to two major data and model-related challenges. To address this, we propose a novel framework called CoD-MTL leveraging multi-task learning to model the semantic relationships between various CoD prediction tasks jointly. Specifically, we develop a novel tree distillation strategy for multi-task learning, which combines the strength of both the tree model and multi-task learning. Experimental results are presented to show the precise and reliable CoD predictions of our framework. A case study is conducted to demonstrate the clinical importance of our method in the liver transplant. 1 Introduction Organ transplant is a crucial therapeutic option for individuals with end-stage diseases, e.g., kidney failure [1], liver failure [2], liver cancer [3], etc.
Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection
Madan, Neelu, Ristea, Nicolae-Catalin, Ionescu, Radu Tudor, Nasrollahi, Kamal, Khan, Fahad Shahbaz, Moeslund, Thomas B., Shah, Mubarak
Anomaly detection has recently gained increasing attention in the field of computer vision, likely due to its broad set of applications ranging from product fault detection on industrial production lines and impending event detection in video surveillance to finding lesions in medical scans. Regardless of the domain, anomaly detection is typically framed as a one-class classification task, where the learning is conducted on normal examples only. An entire family of successful anomaly detection methods is based on learning to reconstruct masked normal inputs (e.g. patches, future frames, etc.) and exerting the magnitude of the reconstruction error as an indicator for the abnormality level. Unlike other reconstruction-based methods, we present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. The proposed self-supervised block is extremely flexible, enabling information masking at any layer of a neural network and being compatible with a wide range of neural architectures. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss. Furthermore, we show that our block is applicable to a wider variety of tasks, adding anomaly detection in medical images and thermal videos to the previously considered tasks based on RGB images and surveillance videos. We exhibit the generality and flexibility of SSMCTB by integrating it into multiple state-of-the-art neural models for anomaly detection, bringing forth empirical results that confirm considerable performance improvements on five benchmarks. We release our code and data as open source at: https://github.com/ristea/ssmctb.
Enhancing Ayurvedic Diagnosis using Multinomial Naive Bayes and K-modes Clustering: An Investigation into Prakriti Types and Dosha Overlapping
Bidve, Pranav, Mishra, Shalini, J, Annapurna
The identification of Prakriti types for the human body is a long-lost medical practice in finding the harmony between the nature of human beings and their behaviour. There are 3 fundamental Prakriti types of individuals. A person can belong to any Dosha. In the existing models, researchers have made use of SVM, KNN, PCA, Decision Tree, and various other algorithms. The output of these algorithms was quite decent, but it can be enhanced with the help of Multinomial Naive Bayes and K-modes clustering. Most of the researchers have confined themselves to 3 basic classes. This might not be accurate in the real-world scenario, where overlapping might occur. Considering these, we have classified the Doshas into 7 categories, which includes overlapping of Doshas. These are namely, VATT-Dosha, PITT-Dosha, KAPH-Dosha, VATT-PITT-Dosha, PITT-KAPH-Dosha, KAPH-VATT-Dosha, and VATT-PITT-KAPH-Dosha. The data used contains a balanced set of all individual entries on which preprocessing steps of machine learning have been performed. Chi-Square test for handling categorical data is being used for feature selection. For model fitting, the method used in this approach is K-modes clustering. The empirical results demonstrate a better result while using the MNB classifier. All key findings of this work have achieved 0.90 accuracy, 0.81 precision, 0.91 F-score, and 0.90 recall. The discussion suggests a provident analysis of the seven clusters and predicts their occurrence. The results have been consolidated to improve the Ayurvedic advancements with machine learning.
SFUSNet: A Spatial-Frequency domain-based Multi-branch Network for diagnosis of Cervical Lymph Node Lesions in Ultrasound Images
Yue, Yubiao, Xue, Jun, Liang, Haihua, Luo, Bingchun, Li, Zhenzhang
Booming deep learning has substantially improved the diagnosis for diverse lesions in ultrasound images, but a conspicuous research gap concerning cervical lymph node lesions still remains. The objective of this work is to diagnose cervical lymph node lesions in ultrasound images by leveraging a deep learning model. To this end, we first collected 3392 cervical ultrasound images containing normal lymph nodes, benign lymph node lesions, malignant primary lymph node lesions, and malignant metastatic lymph node lesions. Given that ultrasound images are generated by the reflection and scattering of sound waves across varied bodily tissues, we proposed the Conv-FFT Block. It integrates convolutional operations with the fast Fourier transform to more astutely model the images. Building upon this foundation, we designed a novel architecture, named SFUSNet. SFUSNet not only discerns variances in ultrasound images from the spatial domain but also adeptly captures micro-structural alterations across various lesions in the frequency domain. To ascertain the potential of SFUSNet, we benchmarked it against 12 popular architectures through five-fold cross-validation. The results show that SFUSNet is the state-of-the-art model and can achieve 92.89% accuracy. Moreover, its average precision, average sensitivity and average specificity for four types of lesions achieve 90.46%, 89.95% and 97.49%, respectively.
Paying Attention to Astronomical Transients: Introducing the Time-series Transformer for Photometric Classification
Allam, Tarek Jr., McEwen, Jason D.
Future surveys such as the Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will observe an order of magnitude more astrophysical transient events than any previous survey before. With this deluge of photometric data, it will be impossible for all such events to be classified by humans alone. Recent efforts have sought to leverage machine learning methods to tackle the challenge of astronomical transient classification, with ever improving success. Transformers are a recently developed deep learning architecture, first proposed for natural language processing, that have shown a great deal of recent success. In this work we develop a new transformer architecture, which uses multi-head self attention at its core, for general multi-variate time-series data. Furthermore, the proposed time-series transformer architecture supports the inclusion of an arbitrary number of additional features, while also offering interpretability. We apply the time-series transformer to the task of photometric classification, minimising the reliance of expert domain knowledge for feature selection, while achieving results comparable to state-of-the-art photometric classification methods. We achieve a logarithmic-loss of 0.507 on imbalanced data in a representative setting using data from the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC). Moreover, we achieve a micro-averaged receiver operating characteristic area under curve of 0.98 and micro-averaged precision-recall area under curve of 0.87.
Conformal Predictions for Longitudinal Data
Batra, Devesh, Mercuri, Salvatore, Khraishi, Raad
The improvement in predictive performance of machine learning models over the last two decades have made them essential components of decision-making pipelines across high-stake domains such as medicine and finance. However, the point estimates yielded by these predictive models are insufficient in these critical application domains, where uncertainty estimates are of particular interest for informed decision-making (see Harries et al. (1999); Dรญaz-Gonzรกlez et al. (2012); Mears et al. (2015) for uncertainty quantification in these domains). While post-hoc methods such as bootstrapping, jackknife and other ensembling procedures (see Alaa et al. (2020); van der Schaar et al. (2020); Xu & Xie (2020)) are popularly used for uncertainty estimation of a particular statistic (such as model metrics), they are only able to provide theoretical guarantees under additional assumptions on the underlying model and data distribution. This limitation, however, is addressed by the conformal prediction framework, which provides a principled way to perform model-agnostic and distribution-free uncertainty quantification of complex machine learning models. Conformal predictions are a powerful tool for constructing prediction sets or intervals that provide reliable coverage guarantees for the true value. These guarantees are typically based on the assumption of data exchangeability, which is often violated in time series data due to temporal dependencies and non-stationarity. However, accurate uncertainty quantification is crucial for time series data, which is central to applications ranging from medical diagnosis to energy demand and stock market forecasting. Consequently, there has been a growing interest in developing conformal prediction methods that can handle non-exchangeable data.
An experimental system for detection and localization of hemorrhage using ultra-wideband microwaves with deep learning
Hedayati, Eisa, Safari, Fatemeh, Verghese, George, Ciancia, Vito R., Sodickson, Daniel K., Dehkharghani, Seena, Alon, Leeor
Stroke is a leading cause of mortality and disability. Emergent diagnosis and intervention are critical, and predicated upon initial brain imaging; however, existing clinical imaging modalities are generally costly, immobile, and demand highly specialized operation and interpretation. Low-energy microwaves have been explored as low-cost, small form factor, fast, and safe probes of tissue dielectric properties, with both imaging and diagnostic potential. Nevertheless, challenges inherent to microwave reconstruction have impeded progress, hence microwave imaging (MWI) remains an elusive scientific aim. Herein, we introduce a dedicated experimental framework comprising a robotic navigation system to translate blood-mimicking phantoms within an anatomically realistic human head model. An 8-element ultra-wideband (UWB) array of modified antipodal Vivaldi antennas was developed and driven by a two-port vector network analyzer spanning 0.6-9.0 GHz at an operating power of 1 mw. Complex scattering parameters were measured, and dielectric signatures of hemorrhage were learned using a dedicated deep neural network for prediction of hemorrhage classes and localization. An overall sensitivity and specificity for detection >0.99 was observed, with Rayliegh mean localization error of 1.65 mm. The study establishes the feasibility of a robust experimental model and deep learning solution for UWB microwave stroke detection.
FLEDGE: Ledger-based Federated Learning Resilient to Inference and Backdoor Attacks
Castillo, Jorge, Rieger, Phillip, Fereidooni, Hossein, Chen, Qian, Sadeghi, Ahmad
Federated learning (FL) is a distributed learning process that uses a trusted aggregation server to allow multiple parties (or clients) to collaboratively train a machine learning model without having them share their private data. Recent research, however, has demonstrated the effectiveness of inference and poisoning attacks on FL. Mitigating both attacks simultaneously is very challenging. State-of-the-art solutions have proposed the use of poisoning defenses with Secure Multi-Party Computation (SMPC) and/or Differential Privacy (DP). However, these techniques are not efficient and fail to address the malicious intent behind the attacks, i.e., adversaries (curious servers and/or compromised clients) seek to exploit a system for monetization purposes. To overcome these limitations, we present a ledger-based FL framework known as FLEDGE that allows making parties accountable for their behavior and achieve reasonable efficiency for mitigating inference and poisoning attacks. Our solution leverages crypto-currency to increase party accountability by penalizing malicious behavior and rewarding benign conduct. We conduct an extensive evaluation on four public datasets: Reddit, MNIST, Fashion-MNIST, and CIFAR-10. Our experimental results demonstrate that (1) FLEDGE provides strong privacy guarantees for model updates without sacrificing model utility; (2) FLEDGE can successfully mitigate different poisoning attacks without degrading the performance of the global model; and (3) FLEDGE offers unique reward mechanisms to promote benign behavior during model training and/or model aggregation.
OOD Aware Supervised Contrastive Learning
Seifi, Soroush, Reino, Daniel Olmeda, Chumerin, Nikolay, Aljundi, Rahaf
Out-of-Distribution (OOD) detection is a crucial problem for the safe deployment of machine learning models identifying samples that fall outside of the training distribution, i.e. in-distribution data (ID). Most OOD works focus on the classification models trained with Cross Entropy (CE) and attempt to fix its inherent issues. In this work we leverage powerful representation learned with Supervised Contrastive (SupCon) training and propose a holistic approach to learn a classifier robust to OOD data. We extend SupCon loss with two additional contrast terms. The first term pushes auxiliary OOD representations away from ID representations without imposing any constraints on similarities among auxiliary data. The second term pushes OOD features far from the existing class prototypes, while pushing ID representations closer to their corresponding class prototype. When auxiliary OOD data is not available, we propose feature mixing techniques to efficiently generate pseudo-OOD features. Our solution is simple and efficient and acts as a natural extension of the closed-set supervised contrastive representation learning. We compare against different OOD detection methods on the common benchmarks and show state-of-the-art results.
Quantifying and mitigating the impact of label errors on model disparity metrics
Adebayo, Julius, Hall, Melissa, Yu, Bowen, Chern, Bobbie
Errors in labels obtained via human annotation adversely affect a model's performance. Existing approaches propose ways to mitigate the effect of label error on a model's downstream accuracy, yet little is known about its impact on a model's disparity metrics We empirically characterize how varying levels of label error, in both training and test data, affect these disparity metrics. We find that group calibration and other metrics are sensitive to train-time and test-time label error--particularly for minority groups. This disparate effect persists even for models trained with noise-aware algorithms. To mitigate the impact of training-time label error, we present an approach to estimate the influence of a training input's label on a model's group disparity metric. We empirically assess the proposed approach on a variety of datasets and find significant improvement, compared to alternative approaches, in identifying training inputs that improve a model's disparity metric. We complement the approach with an automatic relabel-and-finetune scheme that produces updated models with, provably, improved group calibration error. Label error (noise) -- mistakes associated with the label assigned to a data point -- is a pervasive problem in machine learning (Northcutt et al., 2021). For example, 30 percent of a random 1000 samples from the Google Emotions dataset (Demszky et al., 2020) had label errors (Chen, 2022). Similarly, an analysis of the MS COCO dataset found that up to 37 percent (273,834 errors) of all annotations are erroneous (Murdoch, 2022). Yet, little is known about the effect of label error on a model's group-based disparity metrics like equal odds (Hardt et al., 2016), group calibration (Pleiss et al., 2017), and false positive rate (Barocas et al., 2019). It is now common practice to conduct'fairness' audits (see: (Buolamwini and Gebru, 2018; Raji and Buolamwini, 2019; Bakalar et al., 2021)) of a model's predictions to identify data subgroups where the model underperforms. Label error in the test data used to conduct a fairness audit renders the results unreliable. Similarly, label error in the training data, especially if the error is systematically more prevalent in certain groups, can lead to models that associate erroneous labels to such groups. The reliability of a fairness audit rests on the assumption that labels are accurate; yet, the sensitivity of a model's disparity metrics to label error is still poorly understood. Towards such end, we ask: what is the effect of label error on a model's disparity metric? We address the high-level question in a two-pronged manner via the following questions: 1. Research Question 1: What is the sensitivity of a model's disparity metric to label errors in training and test data? Does the effect of label error vary based on group size? 2. Research Question 2: How can a practitioner identify training points whose labels have the most influence on a model's group disparity metric?