Performance Analysis
Benchmarking Early Agitation Prediction in Community-Dwelling People with Dementia Using Multimodal Sensors and Machine Learning
Abedi, Ali, Chu, Charlene H., Khan, Shehroz S.
Agitation is one of the most common responsive behaviors in people living with dementia, particularly among those residing in community settings without continuous clinical supervision. Timely prediction of agitation can enable early intervention, reduce caregiver burden, and improve the quality of life for both patients and caregivers. This study aimed to develop and benchmark machine learning approaches for the early prediction of agitation in community-dwelling older adults with dementia using multimodal sensor data. A new set of agitation-related contextual features derived from activity data was introduced and employed for agitation prediction. A wide range of machine learning and deep learning models was evaluated across multiple problem formulations, including binary classification for single-timestamp tabular sensor data and multi-timestamp sequential sensor data, as well as anomaly detection for single-timestamp tabular sensor data. The study utilized the Technology Integrated Health Management (TIHM) dataset, the largest publicly available dataset for remote monitoring of people living with dementia, comprising 2,803 days of in-home activity, physiology, and sleep data. The most effective setting involved binary classification of sensor data using the current 6-hour timestamp to predict agitation at the subsequent timestamp. Incorporating additional information, such as time of day and agitation history, further improved model performance, with the highest AUC-ROC of 0.9720 and AUC-PR of 0.4320 achieved by the light gradient boosting machine. This work presents the first comprehensive benchmarking of state-of-the-art techniques for agitation prediction in community-based dementia care using privacy-preserving sensor data. The approach enables accurate, explainable, and efficient agitation prediction, supporting proactive dementia care and aging in place.
Facial Foundational Model Advances Early Warning of Coronary Artery Disease from Live Videos with DigitalShadow
Zhou, Juexiao, Han, Zhongyi, Xin, Mankun, He, Xingwei, Wang, Guotao, Song, Jiaoyan, Luo, Gongning, He, Wenjia, Li, Xintong, Chu, Yuetan, Chen, Juanwen, Wang, Bo, Wu, Xia, Duan, Wenwen, Guo, Zhixia, Bai, Liyan, Pan, Yilin, Bi, Xuefei, Liu, Lu, Feng, Long, He, Xiaonan, Gao, Xin
Abstract--Global population aging presents increasing challenges to healthcare systems, with coronary artery disease (CAD) responsible for approximately 17.8 million deaths annually, making it a leading cause of global mortality . As CAD is largely preventable, early detection and proactive management are essential. In this work, we introduce DigitalShadow, an advanced early warning system for CAD, powered by a fine-tuned facial foundation model. The system is pre-trained on 21 million facial images and subsequently fine-tuned into LiveCAD, a specialized CAD risk assessment model trained on 7,004 facial images from 1,751 subjects across four hospitals in China. DigitalShadow functions passively and contactlessly, extracting facial features from live video streams without requiring active user engagement. Integrated with a personalized database, it generates natural language risk reports and individualized health recommendations. With privacy as a core design principle, DigitalShadow supports local deployment to ensure secure handling of user data. The world's population is rapidly ageing [1], with significant implications for the prevalence of chronic diseases such as Coronary Artery Disease (CAD) [2], affecting not only individuals but also families and societies at large [3]. The number of older people is increasing at an unprecedented rate, projected to grow from approximately 761 million in 2021 to 1.6 billion by 2050, which would represent nearly 16% of the global population, according to the UN's W orld Social Report 2023 [4]. The aging population presents numerous challenges, including increased pressure on healthcare systems, pension schemes, and long-term care facilities, alongside potential economic consequences, which together fuel growing demand for healthcare services [5], [6]. With advancing age, people become more vulnerable to various critical diseases [7], such as CAD [8], stroke [9], cancer [10], and Parkinson's disease (PD) [11], [12], [13], leading to considerable morbidity and mortality [14].
Nonlinear Causal Discovery through a Sequential Edge Orientation Approach
Recent advances have established the identifiability of a directed acyclic graph (DAG) under additive noise models (ANMs), spurring the development of various causal discovery methods. However, most existing methods make restrictive model assumptions, rely heavily on general independence tests, or require substantial computational time. To address these limitations, we propose a sequential procedure to orient undirected edges in a completed partial DAG (CPDAG), representing an equivalence class of DAGs, by leveraging the pairwise additive noise model (PANM) to identify their causal directions. We prove that this procedure can recover the true causal DAG assuming a restricted ANM. Building on this result, we develop a novel constraint-based algorithm for learning causal DAGs under nonlinear ANMs. Given an estimated CPDAG, we develop a ranking procedure that sorts undirected edges by their adherence to the PANM, which defines an evaluation order of the edges. To determine the edge direction, we devise a statistical test that compares the log-likelihood values, evaluated with respect to the competing directions, of a sub-graph comprising just the candidate nodes and their identified parents in the partial DAG. We further establish the structural learning consistency of our algorithm in the large-sample limit. Extensive experiments on synthetic and real-world datasets demonstrate that our method is computationally efficient, robust to model misspecification, and consistently outperforms many existing nonlinear DAG learning methods.
Neural Responses to Affective Sentences Reveal Signatures of Depression
Kommineni, Aditya, Jeong, Woojae, Avramidis, Kleanthis, McDaniel, Colin, Hughes, Myzelle, McGee, Thomas, Kaiser, Elsi, Lerman, Kristina, Blank, Idan A., Byrd, Dani, Habibi, Assal, Cahn, B. Rael, Kadiri, Sudarsana, Medani, Takfarinas, Leahy, Richard M., Narayanan, Shrikanth
Depression is one of the most prevalent mental health disorders worldwide, with estimates indicating that around 5% of the worlds' adult population [1, 2] suffers from this condition. The primary methods for screening and monitoring depression rely on self-reported questionnaires, such as the Patient Health Questionnaire (PHQ-9) [3], Beck's Depression Inventory (BDI) [4] and Hamilton Depression Ratings Scale (HDRS) [5]. While these questionnaires are effective to varying degrees at screening patients for depression, they provide only limited information about the affected underlying neuro-cognitive processes in individuals, limiting the ability to personalize treatments. Given the heterogeneity of depressive symptomatology across patient populations [6, 7], it is crucial to elucidate the underlying neurophysiological mechanisms to support the development of more effective and individualized procedures for screening, monitoring, and treatment. Prior functional imaging studies have identified increased activity in anterior cin-gulate cortex (especially the subgenual anterior cingulate) during presentation of emotional stimuli, altered connectivity in prefrontal cortical areas, and default mode network as potential differentiating markers in depressed participants [8-13].
Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completeness
Wang, Cheng-Long, Li, Qi, Xiang, Zihang, Cao, Yinzhi, Wang, Di
Growing concerns over data privacy and security highlight the importance of machine unlearning--removing specific data influences from trained models without full retraining. Techniques like Membership Inference Attacks (MIAs) are widely used to externally assess successful unlearning. However, existing methods face two key limitations: (1) maximizing MIA effectiveness (e.g., via online attacks) requires prohibitive computational resources, often exceeding retraining costs; (2) MIAs, designed for binary inclusion tests, struggle to capture granular changes in approximate unlearning. To address these challenges, we propose the Interpolated Approximate Measurement (IAM), a framework natively designed for unlearning inference. IAM quantifies sample-level unlearning completeness by interpolating the model's generalization-fitting behavior gap on queried samples. IAM achieves strong performance in binary inclusion tests for exact unlearning and high correlation for approximate unlearning--scalable to LLMs using just one pre-trained shadow model. We theoretically analyze how IAM's scoring mechanism maintains performance efficiently. We then apply IAM to recent approximate unlearning algorithms, revealing general risks of both over-unlearning and under-unlearning, underscoring the need for stronger safeguards in approximate unlearning systems. The code is available at https://github.com/Happy2Git/Unlearning_Inference_IAM.
Hey, That's My Data! Label-Only Dataset Inference in Large Language Models
Xiong, Chen, Wang, Zihao, Zhu, Rui, Ho, Tsung-Yi, Chen, Pin-Yu, Xiong, Jingwei, Tang, Haixu, Ohno-Machado, Lucila
Large Language Models (LLMs) have revolutionized Natural Language Processing by excelling at interpreting, reasoning about, and generating human language. However, their reliance on large-scale, often proprietary datasets poses a critical challenge: unauthorized usage of such data can lead to copyright infringement and significant financial harm. Existing dataset-inference methods typically depend on log probabilities to detect suspicious training material, yet many leading LLMs have begun withholding or obfuscating these signals. This reality underscores the pressing need for label-only approaches capable of identifying dataset membership without relying on internal model logits. We address this gap by introducing CatShift, a label-only dataset-inference framework that capitalizes on catastrophic forgetting: the tendency of an LLM to overwrite previously learned knowledge when exposed to new data. If a suspicious dataset was previously seen by the model, fine-tuning on a portion of it triggers a pronounced post-tuning shift in the model's outputs; conversely, truly novel data elicits more modest changes. By comparing the model's output shifts for a suspicious dataset against those for a known non-member validation set, we statistically determine whether the suspicious set is likely to have been part of the model's original training corpus. Extensive experiments on both open-source and API-based LLMs validate CatShift's effectiveness in logit-inaccessible settings, offering a robust and practical solution for safeguarding proprietary data.
When to Trust Context: Self-Reflective Debates for Context Reliability
Zhou, Zeqi, Wu, Fang, Talaei, Shayan, Zhao, Haokai, Meixin, Cheng, Xu, Tinson, Saberi, Amin, Choi, Yejin
Large language models frequently encounter conflicts between their parametric knowledge and contextual input, often resulting in factual inconsistencies or hallucinations. We propose Self-Reflective Debate for Contextual Reliability (SR-DCR), a lightweight framework that integrates token-level self-confidence with an asymmetric multi-agent debate to adjudicate such conflicts. A critic, deprived of context, challenges a defender who argues from the given passage; a judge model evaluates the debate and determines the context's reliability. The final answer is selected by combining the verdict with model confidence. Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently enhances robustness to misleading context while maintaining accuracy on trustworthy inputs, outperforming both classical debate and confidence-only baselines with minimal computational overhead. The code is available at https://github.com/smiles724/Self-Reflective-Debates.
Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Models
Zhu, Chaoyi, Li, Zaitang, Yang, Renyi, Birke, Robert, Chen, Pin-Yu, Ho, Tsung-Yi, Chen, Lydia Y.
Watermarking becomes one of the pivotal solutions to trace and verify the origin of synthetic images generated by artificial intelligence models, but it is not free of risks. Recent studies demonstrate the capability to forge watermarks from a target image onto cover images via adversarial optimization without knowledge of the target generative model and watermark schemes. In this paper, we uncover a greater risk of an optimization-free and universal watermark forgery that harnesses existing regenerative diffusion models. Our proposed forgery attack, PnP (Plug-and-Plant), seamlessly extracts and integrates the target watermark via regenerating the image, without needing any additional optimization routine. It allows for universal watermark forgery that works independently of the target image's origin or the watermarking model used. We explore the watermarked latent extracted from the target image and visual-textual context of cover images as priors to guide sampling of the regenerative process. Extensive evaluation on 24 scenarios of model-data-watermark combinations demonstrates that PnP can successfully forge the watermark (up to 100% detectability and user attribution), and maintain the best visual perception. By bypassing model retraining and enabling adaptability to any image, our approach significantly broadens the scope of forgery attacks, presenting a greater challenge to the security of current watermarking techniques for diffusion models and the authority of watermarking schemes in synthetic data generation and governance.
Bootstrapping World Models from Dynamics Models in Multimodal Foundation Models
Qiu, Yifu, Ziser, Yftah, Korhonen, Anna, Cohen, Shay B., Ponti, Edoardo M.
To what extent do vision-and-language foundation models possess a realistic world model (observation $\times$ action $\rightarrow$ observation) and a dynamics model (observation $\times$ observation $\rightarrow$ action), when actions are expressed through language? While open-source foundation models struggle with both, we find that fine-tuning them to acquire a dynamics model through supervision is significantly easier than acquiring a world model. In turn, dynamics models can be used to bootstrap world models through two main strategies: 1) weakly supervised learning from synthetic data and 2) inference time verification. Firstly, the dynamics model can annotate actions for unlabelled pairs of video frame observations to expand the training data. We further propose a new objective, where image tokens in observation pairs are weighted by their importance, as predicted by a recognition model. Secondly, the dynamics models can assign rewards to multiple samples of the world model to score them, effectively guiding search at inference time. We evaluate the world models resulting from both strategies through the task of action-centric image editing on Aurora-Bench. Our best model achieves a performance competitive with state-of-the-art image editing models, improving on them by a margin of $15\%$ on real-world subsets according to GPT4o-as-judge, and achieving the best average human evaluation across all subsets of Aurora-Bench.
What Really is a Member? Discrediting Membership Inference via Poisoning
Mangaokar, Neal, Hooda, Ashish, Li, Zhuohang, Malin, Bradley A., Fawaz, Kassem, Jha, Somesh, Prakash, Atul, Chowdhury, Amrita Roy
Membership inference tests aim to determine whether a particular data point was included in a language model's training set. However, recent works have shown that such tests often fail under the strict definition of membership based on exact matching, and have suggested relaxing this definition to include semantic neighbors as members as well. In this work, we show that membership inference tests are still unreliable under this relaxation - it is possible to poison the training dataset in a way that causes the test to produce incorrect predictions for a target point. We theoretically reveal a trade-off between a test's accuracy and its robustness to poisoning. We also present a concrete instantiation of this poisoning attack and empirically validate its effectiveness. Our results show that it can degrade the performance of existing tests to well below random.