Goto

Collaborating Authors

 facial expression


Y ouTubePD: A Multimodal Benchmark for Parkinson's Disease Analysis Supplementary Material

Neural Information Processing Systems

We include all our annotations and extracted landmarks. This ensures that we uphold the highest standards of ethical data usage. In Table A1, we summarize the severity label distribution in Y ouTubePD. We also summarize the demographic distribution in Y ouTubePD, split between PD-positive and healthy control (HC), or PD-negative, subjects. This decision is based on the clinician's suggestion, since an accurate UPDRS facial expression rating would require more This strategy also allows for a finer classification.




Scientists find clues in your facial expressions that could be a hidden sign of autism

Daily Mail - Science & tech

Woke wannabe LA mayor melts down during radio interview, says she deserves job because she's a MOTHER - then gets her own age wrong I got the'taboo' cancer soaring among women. Treatment saved my life... but I can NEVER have sex again. It didn't have to be like this AMANDA PLATELL: This single line in Brooklyn Beckham's nuclear outburst is brutal... but it's made me rethink EVERYTHING about Victoria and David The bitter trademark row at the heart of the Beckham feud: Why'devastated' Victoria bore the brunt of Brooklyn's eviscerating statement that has left her'on the floor in pieces' Cut BACK on breakfast cereal. Nick Reiner is'childlike' in jail and so out of it he cannot process the murders of his parents, insider claims The Osteopenia Plague: Almost HALF of over-50s now have the dreaded bone disease. Dark side of America's favorite vacation hotspot... where women are subjected to the most horrific sex attacks imaginable Disturbing video appears to show former Disney star shoving his ex-fiancée after'hammer threat'... as Matt Prokop is arrested for child pornography Joseph Gordon-Levitt was the hottest actor in Hollywood... then vanished: Unearthing family tragedy that sparked disappearance and has left'lasting' scars Shades-wearing Macron hits back at'bully' Trump and warns'we're shifting to a world without rules' where'international law is trampled underfoot and the only law that matters is that of the strongest' Trump reveals why he leaked world leaders' messages and his secret role in foiling a prison break in Syria: Live updates Brooklyn Beckham and Nicola Peltz's wedding guest speaks out and claims Victoria DID dance inappropriately with her son A person's facial reactions may reveal if they have autism, as scientists have found that those with the condition'speak a different language' with their expressions.


Psychiatry has finally found an objective way to spot mental illness

New Scientist

"It seems like this past week has been quite challenging for you," a disembodied voice tells me, before proceeding to ask a series of increasingly personal questions. "Have you been feeling down or depressed?" "Can you describe what this feeling has been like for you?" "Does the feeling lift at all when something good happens?" When I respond to each one, my chatbot interviewer thanks me for my honesty and empathises with any issues. By the end of the conversation, I will have also spoken about my sleep patterns, sex drive and appetite for food.


Generative Neural Articulated Radiance Fields

Neural Information Processing Systems

Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress. These 3D GANs, however, have not been demonstrated for human bodies and the generated radiance fields of existing frameworks are not directly editable, limiting their applicability in downstream tasks. We propose a solution to these challenges by developing a 3D GAN framework that learns to generate radiance fields of human bodies or faces in a canonical pose and warp them using an explicit deformation field into a desired body pose or facial expression. Using our framework, we demonstrate the first high-quality radiance field generation results for human bodies. Moreover, we show that our deformation-aware training procedure significantly improves the quality of generated bodies or faces when editing their poses or facial expressions compared to a 3D GAN that is not trained with explicit deformations.


Deception Detection in Dyadic Exchanges Using Multimodal Machine Learning: A Study on a Swedish Cohort

Samuels, Thomas Jack, Rugolon, Franco, Hau, Stephan, Högman, Lennart

arXiv.org Artificial Intelligence

This study investigates the efficacy of using multimodal machine learning techniques to detect deception in dyadic interactions, focusing on the integration of data from both the deceiver and the deceived. We compare early and late fusion approaches, utilizing audio and video data - specifically, Action Units and gaze information - across all possible combinations of modalities and participants. Our dataset, newly collected from Swedish native speakers engaged in truth or lie scenarios on emotionally relevant topics, serves as the basis for our analysis. The results demonstrate that incorporating both speech and facial information yields superior performance compared to single-modality approaches. Moreover, including data from both participants significantly enhances deception detection accuracy, with the best performance (71%) achieved using a late fusion strategy applied to both modalities and participants. These findings align with psychological theories suggesting differential control of facial and vocal expressions during initial interactions. As the first study of its kind on a Scandinavian cohort, this research lays the groundwork for future investigations into dyadic interactions, particularly within psychotherapy settings.


Classification of User Satisfaction in HRI with Social Signals in the Wild

Schiffmann, Michael, Jeschke, Sabina, Richert, Anja

arXiv.org Artificial Intelligence

Socially interactive agents (SIAs) are being used in various scenarios and are nearing productive deployment. Evaluating user satisfaction with SIAs' performance is a key factor in designing the interaction between the user and SIA. Currently, subjective user satisfaction is primarily assessed manually through questionnaires or indirectly via system metrics. This study examines the automatic classification of user satisfaction through analysis of social signals, aiming to enhance both manual and autonomous evaluation methods for SIAs. During a field trial at the Deutsches Museum Bonn, a Furhat Robotics head was employed as a service and information hub, collecting an "in-the-wild" dataset. This dataset comprises 46 single-user interactions, including questionnaire responses and video data. Our method focuses on automatically classifying user satisfaction based on time series classification. We use time series of social signal metrics derived from the body pose, time series of facial expressions, and physical distance. This study compares three feature engineering approaches on different machine learning models. The results confirm the method's effectiveness in reliably identifying interactions with low user satisfaction without the need for manually annotated datasets. This approach offers significant potential for enhancing SIA performance and user experience through automated feedback mechanisms.


"Why the face?": Exploring Robot Error Detection Using Instrumented Bystander Reactions

Parreira, Maria Teresa, Zhang, Ruidong, Lingaraju, Sukruth Gowdru, Bremers, Alexandra, Fang, Xuanyu, Ramirez-Aristizabal, Adolfo, Saha, Manaswi, Kuniavsky, Michael, Zhang, Cheng, Ju, Wendy

arXiv.org Artificial Intelligence

How do humans recognize and rectify social missteps? We achieve social competence by looking around at our peers, decoding subtle cues from bystanders - a raised eyebrow, a laugh - to evaluate the environment and our actions. Robots, however, struggle to perceive and make use of these nuanced reactions. By employing a novel neck-mounted device that records facial expressions from the chin region, we explore the potential of previously untapped data to capture and interpret human responses to robot error. First, we develop NeckNet-18, a 3D facial reconstruction model to map the reactions captured through the chin camera onto facial points and head motion. We then use these facial responses to develop a robot error detection model which outperforms standard methodologies such as using OpenFace or video data, generalizing well especially for within-participant data. Through this work, we argue for expanding human-in-the-loop robot sensing, fostering more seamless integration of robots into diverse human environments, pushing the boundaries of social cue detection and opening new avenues for adaptable robotics.


MF-GCN: A Multi-Frequency Graph Convolutional Network for Tri-Modal Depression Detection Using Eye-Tracking, Facial, and Acoustic Features

Rahman, Sejuti, Deb, Swakshar, Chowdhury, MD. Sameer Iqbal, Sourov, MD. Jubair Ahmed, Shamsuddin, Mohammad

arXiv.org Artificial Intelligence

Depression is a prevalent global mental health disorder, characterised by persistent low mood and anhedonia. However, it remains underdiagnosed because current diagnostic methods depend heavily on subjective clinical assessments. To enable objective detection, we introduce a gold standard dataset of 103 clinically assessed participants collected through a tripartite data approach which uniquely integrated eye tracking data with audio and video to give a comprehensive representation of depressive symptoms. Eye tracking data quantifies the attentional bias towards negative stimuli that is frequently observed in depressed groups. Audio and video data capture the affective flattening and psychomotor retardation characteristic of depression. Statistical validation confirmed their significant discriminative power in distinguishing depressed from non depressed groups. We address a critical limitation of existing graph-based models that focus on low-frequency information and propose a Multi-Frequency Graph Convolutional Network (MF-GCN). This framework consists of a novel Multi-Frequency Filter Bank Module (MFFBM), which can leverage both low and high frequency signals. Extensive evaluation against traditional machine learning algorithms and deep learning frameworks demonstrates that MF-GCN consistently outperforms baselines. In binary classification, the model achieved a sensitivity of 0.96 and F2 score of 0.94. For the 3 class classification task, the proposed method achieved a sensitivity of 0.79 and specificity of 0.87 and siginificantly suprassed other models. To validate generalizability, the model was also evaluated on the Chinese Multimodal Depression Corpus (CMDC) dataset and achieved a sensitivity of 0.95 and F2 score of 0.96. These results confirm that our trimodal, multi frequency framework effectively captures cross modal interaction for accurate depression detection.