Performance Analysis
Benchmarking Histopathology Foundation Models for Ovarian Cancer Bevacizumab Treatment Response Prediction from Whole Slide Images
Mallya, Mayur, Mirabadi, Ali Khajegili, Farahani, Hossein, Bashashati, Ali
Bevacizumab is a widely studied targeted therapeutic drug used in conjunction with standard chemotherapy for the treatment of recurrent ovarian cancer. While its administration has shown to increase the progression-free survival (PFS) in patients with advanced stage ovarian cancer, the lack of identifiable biomarkers for predicting patient response has been a major roadblock in its effective adoption towards personalized medicine. In this work, we leverage the latest histopathology foundation models trained on large-scale whole slide image (WSI) datasets to extract ovarian tumor tissue features for predicting bevacizumab response from WSIs. Our extensive experiments across a combination of different histopathology foundation models and multiple instance learning (MIL) strategies demonstrate capability of these large models in predicting bevacizumab response in ovarian cancer patients with the best models achieving an AUC score of 0.86 and an accuracy score of 72.5%. Furthermore, our survival models are able to stratify high- and low-risk cases with statistical significance (p < 0.05) even among the patients with the aggressive subtype of high-grade serous ovarian carcinoma. This work highlights the utility of histopathology foundation models for the task of ovarian bevacizumab response prediction from WSIs. The high-attention regions of the WSIs highlighted by these models not only aid the model explainability but also serve as promising imaging biomarkers for treatment prognosis.
Automated Review Generation Method Based on Large Language Models
Wu, Shican, Ma, Xiao, Luo, Dehui, Li, Lulu, Shi, Xiangcheng, Chang, Xin, Lin, Xiaoyun, Luo, Ran, Pei, Chunlei, Zhao, Zhi-Jian, Gong, Jinlong
Literature research, vital for scientific advancement, is overwhelmed by the vast ocean of available information. Addressing this, we propose an automated review generation method based on Large Language Models (LLMs) to streamline literature processing and reduce cognitive load. In case study on propane dehydrogenation (PDH) catalysts, our method swiftly generated comprehensive reviews from 343 articles, averaging seconds per article per LLM account. Extended analysis of 1041 articles provided deep insights into catalysts' composition, structure, and performance. Recognizing LLMs' hallucinations, we employed a multi-layered quality control strategy, ensuring our method's reliability and effective hallucination mitigation. Expert verification confirms the accuracy and citation integrity of generated reviews, demonstrating LLM hallucination risks reduced to below 0.5% with over 95% confidence. Released Windows application enables one-click review generation, aiding researchers in tracking advancements and recommending literature. This approach showcases LLMs' role in enhancing scientific research productivity and sets the stage for further exploration.
Confidence Estimation for Automatic Detection of Depression and Alzheimer's Disease Based on Clinical Interviews
Wu, Wen, Zhang, Chao, Woodland, Philip C.
It can also facilitate the identification of and depression has attracted increased attention. Confidence estimation ambiguous and borderline cases, necessitating the input of clinical is crucial for a trust-worthy automatic diagnostic system expertise. While confidence estimation techniques have which informs the clinician about the confidence of model been applied in areas like speech recognition [23-26] and dialogue predictions and helps reduce the risk of misdiagnosis. This paper systems [27], their application in detecting mental illnesses investigates confidence estimation for automatic detection through speech analysis remains largely unexplored. of AD and depression based on clinical interviews. A novel Bayesian approach is proposed which uses a dynamic Dirichlet This paper investigates confidence estimation for automatic prior distribution to model the second-order probability of AD and depression detection based on speech recordings from the predictive distribution.
Unleash the Power of Ellipsis: Accuracy-enhanced Sparse Vector Technique with Exponential Noise
Liu, Yuhan, Wang, Sheng, Liu, Yixuan, Li, Feifei, Chen, Hong
The Sparse Vector Technique (SVT) is one of the most fundamental tools in differential privacy (DP). It works as a backbone for adaptive data analysis by answering a sequence of queries on a given dataset, and gleaning useful information in a privacy-preserving manner. Unlike the typical private query releases that directly publicize the noisy query results, SVT is less informative -- it keeps the noisy query results to itself and only reveals a binary bit for each query, indicating whether the query result surpasses a predefined threshold. To provide a rigorous DP guarantee for SVT, prior works in the literature adopt a conservative privacy analysis by assuming the direct disclosure of noisy query results as in typical private query releases. This approach, however, hinders SVT from achieving higher query accuracy due to an overestimation of the privacy risks, which further leads to an excessive noise injection using the Laplacian or Gaussian noise for perturbation. Motivated by this, we provide a new privacy analysis for SVT by considering its less informative nature. Our analysis results not only broaden the range of applicable noise types for perturbation in SVT, but also identify the exponential noise as optimal among all evaluated noises (which, however, is usually deemed non-applicable in prior works). The main challenge in applying exponential noise to SVT is mitigating the sub-optimal performance due to the bias introduced by noise distributions. To address this, we develop a utility-oriented optimal threshold correction method and an appending strategy, which enhances the performance of SVT by increasing the precision and recall, respectively. The effectiveness of our proposed methods is substantiated both theoretically and empirically, demonstrating significant improvements up to $50\%$ across evaluated metrics.
Can I trust my anomaly detection system? A case study based on explainable AI
Rashid, Muhammad, Amparore, Elvio, Ferrari, Enrico, Verda, Damiano
Generative models based on variational autoencoders are a popular technique for detecting anomalies in images in a semi-supervised context. A common approach employs the anomaly score to detect the presence of anomalies, and it is known to reach high level of accuracy on benchmark datasets. However, since anomaly scores are computed from reconstruction disparities, they often obscure the detection of various spurious features, raising concerns regarding their actual efficacy. This case study explores the robustness of an anomaly detection system based on variational autoencoder generative models through the use of eXplainable AI methods. The goal is to get a different perspective on the real performances of anomaly detectors that use reconstruction differences. In our case study we discovered that, in many cases, samples are detected as anomalous for the wrong or misleading factors.
Classification of Alzheimer's Dementia vs. Healthy subjects by studying structural disparities in fMRI Time-Series of DMN
Noble, Sneha, Pradeep, Chakka Sai, Sinha, Neelam, Issac, Thomas Gregor
Time series from different regions of interest (ROI) of default mode network (DMN) from Functional Magnetic Resonance Imaging (fMRI) can reveal significant differences between healthy and unhealthy people. Here, we propose the utility of an existing metric quantifying the lack/presence of structure in a signal called, "deviation from stochasticity" (DS) measure to characterize resting-state fMRI time series. The hypothesis is that differences in the level of structure in the time series can lead to discrimination between the subject groups. In this work, an autoencoder-based model is utilized to learn efficient representations of data by training the network to reconstruct its input data. The proposed methodology is applied on fMRI time series of 50 healthy individuals and 50 subjects with Alzheimer's Disease (AD), obtained from publicly available ADNI database. DS measure for healthy fMRI as expected turns out to be different compared to that of AD. Peak classification accuracy of 95% was obtained using Gradient Boosting classifier, using the DS measure applied on 100 subjects.
Sentiment Analysis of Lithuanian Online Reviews Using Large Language Models
Vileikytฤ, Brigita, Lukoลกeviฤius, Mantas, Stankeviฤius, Lukas
Sentiment analysis is a widely researched area within Natural Language Processing (NLP), attracting significant interest due to the advent of automated solutions. Despite this, the task remains challenging because of the inherent complexity of languages and the subjective nature of sentiments. It is even more challenging for less-studied and less-resourced languages such as Lithuanian. Our review of existing Lithuanian NLP research reveals that traditional machine learning methods and classification algorithms have limited effectiveness for the task. In this work, we address sentiment analysis of Lithuanian five-star-based online reviews from multiple domains that we collect and clean. We apply transformer models to this task for the first time, exploring the capabilities of pre-trained multilingual Large Language Models (LLMs), specifically focusing on fine-tuning BERT and T5 models. Given the inherent difficulty of the task, the fine-tuned models perform quite well, especially when the sentiments themselves are less ambiguous: 80.74% and 89.61% testing recognition accuracy of the most popular one- and five-star reviews respectively. They significantly outperform current commercial state-of-the-art general-purpose LLM GPT-4. We openly share our fine-tuned LLMs online.
Hashing based Contrastive Learning for Virtual Screening
Han, Jin, Hong, Yun, Li, Wu-Jun
Virtual screening (VS) is a critical step in computer-aided drug discovery, aiming to identify molecules that bind to a specific target receptor like protein. Traditional VS methods, such as docking, are often too time-consuming for screening large-scale molecular databases. Recent advances in deep learning have demonstrated that learning vector representations for both proteins and molecules using contrastive learning can outperform traditional docking methods. However, given that target databases often contain billions of molecules, real-valued vector representations adopted by existing methods can still incur significant memory and time costs in VS. To address this problem, in this paper we propose a hashing-based contrastive learning method, called DrugHash, for VS. DrugHash treats VS as a retrieval task that uses efficient binary hash codes for retrieval. In particular, DrugHash designs a simple yet effective hashing strategy to enable end-to-end learning of binary hash codes for both protein and molecule modalities, which can dramatically reduce the memory and time costs with higher accuracy compared with existing methods. Experimental results show that DrugHash can outperform existing methods to achieve state-of-the-art accuracy, with a memory saving of 32$\times$ and a speed improvement of 3.5$\times$.
A data balancing approach towards design of an expert system for Heart Disease Prediction
Karmakar, Rahul, Ghosh, Udita, Pal, Arpita, Dey, Sattwiki, Malik, Debraj, Sain, Priyabrata
Heart disease is a serious global health issue that claims millions of lives every year. Early detection and precise prediction are critical to the prevention and successful treatment of heart related issues. A lot of research utilizes machine learning (ML) models to forecast cardiac disease and obtain early detection. In order to do predictive analysis on "Heart disease health indicators " dataset. We employed five machine learning methods in this paper: Decision Tree (DT), Random Forest (RF), Linear Discriminant Analysis, Extra Tree Classifier, and AdaBoost. The model is further examined using various feature selection (FS) techniques. To enhance the baseline model, we have separately applied four FS techniques: Sequential Forward FS, Sequential Backward FS, Correlation Matrix, and Chi2. Lastly, K means SMOTE oversampling is applied to the models to enable additional analysis. The findings show that when it came to predicting heart disease, ensemble approaches in particular, random forests performed better than individual classifiers. The presence of smoking, blood pressure, cholesterol, and physical inactivity were among the major predictors that were found. The accuracy of the Random Forest and Decision Tree model was 99.83%. This paper demonstrates how machine learning models can improve the accuracy of heart disease prediction, especially when using ensemble methodologies. The models provide a more accurate risk assessment than traditional methods since they incorporate a large number of factors and complex algorithms.
Detecting Unsafe Behavior in Neural Network Imitation Policies for Caregiving Robotics
In this paper, the application of imitation learning in caregiving robotics is explored, aiming at addressing the increasing demand for automated assistance in caring for the elderly and disabled. Leveraging advancements in deep learning and control algorithms, the study focuses on training neural network policies using offline demonstrations. A key challenge addressed is the "Policy Stopping" problem, crucial for enhancing safety in imitation learning-based policies, particularly diffusion policies. Novel solutions proposed include ensemble predictors and adaptations of the normalizing flow-based algorithm for early anomaly detection. Comparative evaluations against anomaly detection methods like VAE and Tran-AD demonstrate superior performance on assistive robotics benchmarks. The paper concludes by discussing the further research in integrating safety models into policy training, crucial for the reliable deployment of neural network policies in caregiving robotics.