Accuracy
Examining Deployment and Refinement of the VIOLA-AI Intracranial Hemorrhage Model Using an Interactive NeoMedSys Platform
Liu, Qinghui, Nesvold, Jon E., Raaum, Hanna, Murugesu, Elakkyen, Røvang, Martin, Maclntosh, Bradley J, Bjørnerud, Atle, Skogen, Karoline
Background: There are many challenges and opportunities in the clinical deployment of AI tools in radiology. The current study describes a radiology software platform called NeoMedSys that can enable efficient deployment and refinements of AI models. We evaluated the feasibility and effectiveness of running NeoMedSys for three months in real-world clinical settings and focused on improvement performance of an in-house developed AI model (VIOLA-AI) designed for intracranial hemorrhage (ICH) detection. Methods: NeoMedSys integrates tools for deploying, testing, and optimizing AI models with a web-based medical image viewer, annotation system, and hospital-wide radiology information systems. A prospective pragmatic investigation was deployed using clinical cases of patients presenting to the largest Emergency Department in Norway (site-1) with suspected traumatic brain injury (TBI) or patients with suspected stroke (site-2). We assessed ICH classification performance as VIOLA-AI encountered new data and underwent pre-planned model retraining. Performance metrics included sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve (AUC). Results: NeoMedSys facilitated iterative improvements in the AI model, significantly enhancing its diagnostic accuracy. Automated bleed detection and segmentation were reviewed in near real-time to facilitate re-training VIOLA-AI. The iterative refinement process yielded a marked improvement in classification sensitivity, rising to 90.3% (from 79.2%), and specificity that reached 89.3% (from 80.7%). The bleed detection ROC analysis for the entire sample demonstrated a high area-under-the-curve (AUC) of 0.949 (from 0.873). Model refinement stages were associated with notable gains, highlighting the value of real-time radiologist feedback.
Next-Depth Lookahead Tree
Lee, Jaeho, Kim, Kangjin, Lee, Gyeong Taek
This paper proposes the Next-Depth Lookahead Tree (NDLT), a single-tree model designed to improve performance by evaluating node splits not only at the node being optimized but also by evaluating the quality of the next depth level. Conventional decision trees (DTs) rely on greedy node-by-node partitioning, which often fails to ensure global optimality of the tree and is prone to local optima when early splits are suboptimal. To overcome this limitation, NDLT employs a next-depth lookahead strategy that jointly considers the immediate impurity reduction at the parent node and the expected impurity reduction at its child nodes. Empirical evaluation on diverse and complex datasets, including high-dimensional and imbalanced cases, demonstrates that NDLT achieves performance comparable to or better than classical DTs and ensemble models such as Random Forests, XGBoost, and Light-GBM. These results show that NDLT preserves the interpretability of a single tree while delivering robust predictive accuracy, making it an effective approach for real-world applications where both transparency and performance are required. These authors contributed equally to this work. Introduction Decision Tree (DT) represents one of the earliest and most established algorithms in the field of machine learning, valued for its simplicity, inter-pretability, and broad applicability [1].
Benefits of Online Tilted Empirical Risk Minimization: A Case Study of Outlier Detection and Robust Regression
Yildirim, Yigit E., Demir, Samet, Dogan, Zafer
Empirical Risk Minimization (ERM) is a foundational framework for supervised learning but primarily optimizes average-case performance, often neglecting fairness and robustness considerations. Tilted Empirical Risk Minimization (TERM) extends ERM by introducing an exponential tilt hyperparameter $t$ to balance average-case accuracy with worst-case fairness and robustness. However, in online or streaming settings where data arrive one sample at a time, the classical TERM objective degenerates to standard ERM, losing tilt sensitivity. We address this limitation by proposing an online TERM formulation that removes the logarithm from the classical objective, preserving tilt effects without additional computational or memory overhead. This formulation enables a continuous trade-off controlled by $t$, smoothly interpolating between ERM ($t \to 0$), fairness emphasis ($t > 0$), and robustness to outliers ($t < 0$). We empirically validate online TERM on two representative streaming tasks: robust linear regression with adversarial outliers and minority-class detection in binary classification. Our results demonstrate that negative tilting effectively suppresses outlier influence, while positive tilting improves recall with minimal impact on precision, all at per-sample computational cost equivalent to ERM. Online TERM thus recovers the full robustness-fairness spectrum of classical TERM in an efficient single-sample learning regime.
Simulating a Bias Mitigation Scenario in Large Language Models
Kiashemshaki, Kiana, Torkamani, Mohammad Jalili, Mahmoudi, Negin, Bilehsavar, Meysam Shirdel
Large Language Models (LLMs) have fundamentally transformed the field of natural language processing; however, their vulnerability to biases presents a notable obstacle that threatens both fairness and trust. This review offers an extensive analysis of the bias landscape in LLMs, tracing its roots and expressions across various NLP tasks. Biases are classified into implicit and explicit types, with particular attention given to their emergence from data sources, architectural designs, and contextual deployments. This study advances beyond theoretical analysis by implementing a simulation framework designed to evaluate bias mitigation strategies in practice. The framework integrates multiple approaches including data curation, debiasing during model training, and post-hoc output calibration and assesses their impact in controlled experimental settings. In summary, this work not only synthesizes existing knowledge on bias in LLMs but also contributes original empirical validation through simulation of mitigation strategies.
H-Alpha Anomalyzer: An Explainable Anomaly Detector for Solar H-Alpha Observations
Khazaei, Mahsa, Ahmadzadeh, Azim, Pevtsov, Alexei, Bertello, Luca, Pevtsov, Alexander
Abstract--The plethora of space-borne and ground-based observatories has provided astrophysicists with an unprecedented volume of data, which can only be processed at scale using advanced computing algorithms. Consequently, ensuring the quality of data fed into machine learning (ML) models is critical. The Hα observations from the GONG network represent one such data stream, producing several observations per minute, 24/7, since 2010. In this study, we introduce a lightweight (non-ML) anomaly-detection algorithm, called H-Alpha Anomalyzer, designed to identify anomalous observations based on user-defined criteria. Unlike many black-box algorithms, our approach highlights exactly which regions triggered the anomaly flag and quantifies the corresponding anomaly likelihood. For our comparative analysis, we also created and released a dataset of 2,000 observations, equally divided between anomalous and non-anomalous cases. Our results demonstrate that the proposed model not only outperforms existing methods but also provides explainability, enabling qualitative evaluation by domain experts. Millions of Hα images are produced by the NSF's Global Oscillation Network Group (GONG, [7]), some of which show anomaly patterns and can't be used by either algorithms or scientists. Detecting these images and removing them from the pipeline is a laborious task, especially on such a large scale. Motivated by this, we aim to analyze different solutions for cleaning corrupt Hα images and do a comprehensive quantitative and qualitative analysis of their performance.
JU-NLP at Touché: Covert Advertisement in Conversational AI-Generation and Detection Strategies
Dutta, Arka, Majumdar, Agrik, Biswas, Sombrata, Das, Dipankar, Bandyopadhyay, Sivaji
This paper proposes a comprehensive framework for the generation of covert advertisements within Conversational AI systems, along with robust techniques for their detection. It explores how subtle promotional content can be crafted within AI-generated responses and introduces methods to identify and mitigate such covert advertising strategies. For generation (Sub-Task~1), we propose a novel framework that leverages user context and query intent to produce contextually relevant advertisements. We employ advanced prompting strategies and curate paired training data to fine-tune a large language model (LLM) for enhanced stealthiness. For detection (Sub-Task~2), we explore two effective strategies: a fine-tuned CrossEncoder (\texttt{all-mpnet-base-v2}) for direct classification, and a prompt-based reformulation using a fine-tuned \texttt{DeBERTa-v3-base} model. Both approaches rely solely on the response text, ensuring practicality for real-world deployment. Experimental results show high effectiveness in both tasks, achieving a precision of 1.0 and recall of 0.71 for ad generation, and F1-scores ranging from 0.99 to 1.00 for ad detection. These results underscore the potential of our methods to balance persuasive communication with transparency in conversational AI.
Evaluating Supervised Learning Models for Fraud Detection: A Comparative Study of Classical and Deep Architectures on Imbalanced Transaction Data
Wang, Chao, Nie, Chuanhao, Liu, Yunbo
Fraud detection remains a critical task in high-stakes domains such as finance and e-commerce, where undetected fraudulent transactions can lead to significant economic losses. In this study, we systematically compare the performance of four supervised learning models - Logistic Regression, Random Forest, Light Gradient Boosting Machine (LightGBM), and a Gated Recurrent Unit (GRU) network - on a large-scale, highly imbalanced online transaction dataset. While ensemble methods such as Random Forest and LightGBM demonstrated superior performance in both overall and class-specific metrics, Logistic Regression offered a reliable and interpretable baseline. The GRU model showed strong recall for the minority fraud class, though at the cost of precision, highlighting a trade-off relevant for real-world deployment. Our evaluation emphasizes not only weighted averages but also per-class precision, recall, and F1-scores, providing a nuanced view of each model's effectiveness in detecting rare but consequential fraudulent activity. The findings underscore the importance of choosing models based on the specific risk tolerance and operational needs of fraud detection systems.
AnoF-Diff: One-Step Diffusion-Based Anomaly Detection for Forceful Tool Use
Lin, Yating, Huang, Zixuan, Yang, Fan, Berenson, Dmitry
Abstract-- Multivariate time-series anomaly detection, which is critical for identifying unexpected events, has been explored in the field of machine learning for several decades. However, directly applying these methods to data from forceful tool use tasks is challenging because streaming sensor data in the real world tends to be inherently noisy, exhibits non-stationary behavior, and varies across different tasks and tools. T o address these challenges, we propose a method, AnoF-Diff, based on the diffusion model to extract force-torque features from time-series data and use force-torque features to detect anomalies. We compare our method with other state-of-the-art methods in terms of F1-score and Area Under the Receiver Operating Characteristic curve (AUROC) on four forceful tool-use tasks, demonstrating that our method has better performance and is more robust to a noisy dataset. We also propose the method of parallel anomaly score evaluation based on one-step diffusion and demonstrate how our method can be used for online anomaly detection in several forceful tool use experiments. I. INTRODUCTION As the development of robot sensing and machine learning technologies accelerates, multivariate time series analysis is becoming more and more critical in the robotics field. Robotic systems usually rely on current time-step sensor data for decision-making and control, which makes it possible to miss the potential temporal patterns over multiple time steps. Additionally, some sensor data, such as force-torque signals, require multiple time steps to capture dynamic behaviors and provide meaningful information.
Limitations of Public Chest Radiography Datasets for Artificial Intelligence: Label Quality, Domain Shift, Bias and Evaluation Challenges
Rafferty, Amy, Ramaesh, Rishi, Rajan, Ajitha
Artificial intelligence has shown significant promise in chest radiography, where deep learning models can approach radiologist-level diagnostic performance. Progress has been accelerated by large public datasets such as MIMIC-CXR, ChestX-ray14, PadChest, and CheXpert, which provide hundreds of thousands of labelled images with pathology annotations. However, these datasets also present important limitations. Automated label extraction from radiology reports introduces errors, particularly in handling uncertainty and negation, and radiologist review frequently disagrees with assigned labels. In addition, domain shift and population bias restrict model generalisability, while evaluation practices often overlook clinically meaningful measures. We conduct a systematic analysis of these challenges, focusing on label quality, dataset bias, and domain shift. Our cross-dataset domain shift evaluation across multiple model architectures revealed substantial external performance degradation, with pronounced reductions in AUPRC and F1 scores relative to internal testing. To assess dataset bias, we trained a source-classification model that distinguished datasets with near-perfect accuracy, and performed subgroup analyses showing reduced performance for minority age and sex groups. Finally, expert review by two board-certified radiologists identified significant disagreement with public dataset labels. Our findings highlight important clinical weaknesses of current benchmarks and emphasise the need for clinician-validated datasets and fairer evaluation frameworks.
Credit Card Fraud Detection
Popova, Iva, Gardi, Hamza A. A.
Iva Popova Hamza A. A. Gardi ETIT - KIT, Germany IIIT at ETIT - KIT, Germany Abstract Credit card fraud remains a significant challenge due to class imbalance and fraudsters mimicking legitimate behavior. This study evaluates five machine learning models - Logistic Regression, Random Forest, XGBoost, K - Nearest Neighbors (KNN), and Multi - Lay er Perceptron (MLP) on a real - world dataset using undersampling, SMOTE, and a hybrid approach. Our models are evaluated on the original imbalanced test set to better reflect real - world performance. Results show that the hybrid method achieves the best bala nce between recall and precision, especially improving MLP and KNN performance. I ntroduction Financial fraud is a significant issue that has been continuously increasing over the past few years due to the ever - growing volume of online transactions conduc ted with credit cards. Credit card fraud (CCF) refers to a type of fraud in which an individual other than the cardholder unlawfully conducts transactions using a card that is stolen, lost, or otherwise misused [ 1 ]. CCF has resulted in billions of dollars in losses for banks and other online payment platforms. According to the Federal Trade Commission (FTC), there were 449,076 reports of CCF in 2024, representing a 7.8% increase from the previous year [ 2 ]. Given this trend, new methods must be employed to c apture patterns and dependencies in the data.