Goto

Collaborating Authors

Designing Embodied Cues for Dialog with Robots

AI Magazine

Of all computational systems, robots are unique in their ability to afford embodied interaction using the wider range of human communicative cues. Research on human communication provides strong evidence that embodied cues, when used effectively, elicit social, cognitive, and task outcomes such as improved learning, rapport, motivation, persuasion, and collaborative task performance. While this connection between embodied cues and key outcomes provides a unique opportunity for design, taking advantage of it requires a deeper understanding of how robots might use these cues effectively and the limitations in the extent to which they might achieve such outcomes through embodied interaction. This article aims to underline this opportunity by providing an overview of key embodied cues and outcomes in human communication and describing a research program that explores how robots might generate high-level social, cognitive, and task outcomes such as learning, rapport, and persuasion using embodied cues such as verbal, vocal, and nonverbal cues.


Designing Embodied Cues for Dialogue with Robots

AI Magazine

Of all computational systems, robots are unique in their ability to afford embodied interaction using the wider range of human communicative cues. Research on human communication provides strong evidence that embodied cues, when used effectively, elicit social, cognitive, and task outcomes such as improved learning, rapport, motivation, persuasion, and collaborative task performance. While this connection between embodied cues and key outcomes provides a unique opportunity for design, taking advantage of it requires a deeper understanding of how robots might use these cues effectively and the limitations in the extent to which they might achieve such outcomes through embodied interaction. This article aims to underline this opportunity by providing an overview of key embodied cues and outcomes in human communication and describing a research program that explores how robots might generate high-level social, cognitive, and task outcomes such as learning, rapport, and persuasion using embodied cues such as verbal, vocal, and nonverbal cues. Such representations vary from physical artifacts (such as tangible interfaces) to biological forms (such as humanlike agents and robots) and offer templates for understanding and interacting with complex computational systems (Ullmer and Ishii 2000, Cassell 2001, Breazeal 2003).


Let's Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings

arXiv.org Machine Learning

To enable more natural face-to-face interactions, conversational agents need to adapt their behavior to their interlocutors. One key aspect of this is generation of appropriate non-verbal behavior for the agent, for example facial gestures, here defined as facial expressions and head movements. Most existing gesture-generating systems do not utilize multi-modal cues from the interlocutor when synthesizing non-verbal behavior. Those that do, typically use deterministic methods that risk producing repetitive and non-vivid motions. In this paper, we introduce a probabilistic method to synthesize interlocutor-aware facial gestures - represented by highly expressive FLAME parameters - in dyadic conversations. Our contributions are: a) a method for feature extraction from multi-party video and speech recordings, resulting in a representation that allows for independent control and manipulation of expression and speech articulation in a 3D avatar; b) an extension to MoGlow, a recent motion-synthesis method based on normalizing flows, to also take multi-modal signals from the interlocutor as input and subsequently output interlocutor-aware facial gestures; and c) a subjective evaluation assessing the use and relative importance of the input modalities. The results show that the model successfully leverages the input from the interlocutor to generate more appropriate behavior. Videos, data, and code available at: https://jonepatr.github.io/lets_face_it.


Mirroring to Build Trust in Digital Assistants

arXiv.org Artificial Intelligence

We describe experiments towards building a conversational digital assistant that considers the preferred conversational style of the user. In particular, these experiments are designed to measure whether users prefer and trust an assistant whose conversational style matches their own. To this end we conducted a user study where subjects interacted with a digital assistant that responded in a way that either matched their conversational style, or did not. Using self-reported personality attributes and subjects' feedback on the interactions, we built models that can reliably predict a user's preferred conversational style.


Building an On-Demand Avatar-Based Health Intervention for Behavior Change

AAAI Conferences

We discuss the design and implementation of the pro- totype of an avatar-based health system aimed at pro- viding people access to an effective behavior change intervention which can help them to find and cultivate motivation to change unhealthy lifestyles. An empathic Embodied Conversational Agent (ECA) delivers the in- tervention. The health dialog is directed by a compu- tational model of Motivational Interviewing, a novel effective face-to-face patient-centered counseling style which respects an individual’s pace toward behavior change. Although conducted on a small sample size, re- sults of a preliminary user study to asses users’ accep- tance of the avatar counselor indicate that the current early version of the system prototype is well accepted by 75% of users.