The overall, long-term goal of our research is to develop a The following is a concrete scenario we will use to motivate fully animated robotic companion for home healthcare that our platform for home healthcare and related STEM research.

As part of the "Survivor Buddy" project, we have created an open source speech translator toolkit which allows written or spoken word from multiple independent controllers to be translated into either a single synthetic voice, synthetic voices for each controller, orunchanged natural voice of each controller. The human controllers can work via the internet or be physically co-located with the Survivor Buddy robot. The toolkit is expected to be of use for exploring voice in general human-robot interaction. The Survivor Buddy project is motivated by our prior work which suggests that a trapped victim of a disaster, or other human who is dependent, will treat a rescue robot as a social medium and that the choice of robotic voice will be important. The robot will be both a medium to the "outside" world and a local, independent entity devoted to the victim Figure 1: View from the Survivor Buddy webcam with subpicture (e.g., a buddy).

Robots are being considered for applications where they serve as proxies for humans interacting with another human,such as emergency response, hostage negotiation, and healthcare. In these domains, the human (“dependent”) is connected to multiple other humans (“controllers”) via the robot proxy for long periods of time. The dependent may want to interact with humans but also to engage the robot as a medium to the World Wide Web. In the future, medical personnel may use the robot for victim assistance and comfort while the rescue team plans and monitors extrication. Other applications include healthcare, where the robot is the link between a patient and a medical provider for intermittent,routine interactions, and hostage negotiation, where police may use a bomb squad robot to talk with and build rapport with the suspect while the SWAT team uses the robot’s sensors to build and maintain situation awareness.Under funding from the National Science Foundation, we are finishing the first year of investigating verbal and nonverbal communication strategies for robots who are serving as proxies for multiple humans interact with the humans who are dependent on them. Our work posits that such a robot would occupy a novel social medium position according to the Computers as Social Actors (CASA) model [Nass,Steuer, and Tauber1994] [Reeves and Nass1996]. Given that teleoperated robots are treated socially, it is unlikely that a rescue robot would be treated as a pure medium even if playing music or videos. Likewise, the limitations of autonomy and the interactions of specialists with the dependent prevent the robot from being a true social actor. Instead, social actor and pure medium are two extremes on the agent identity spectrum, with a social medium occupying a middle position.A social medium would be perceived as a loyal, helpful “go between” who is an advocate for the dependent, rather than a device for accomplishing the goals of multiple controllers(medical specialist, structural engineer, rescue operations official, etc.). To explore the social medium identity,we have built a physical prototype of a Survivor Buddy and are creating autonomous affective behaviors and a social medium toolkit to explore human-robot interaction.

Our goal is to develop an interface for older people to effectively communicate with a robotic assistant so that they can safely remain living in their home. We are devising a multimodal interface since people communicate with one another using a variety of verbal and non-verbal signals, including haptics, i.e., physical interactions. We view haptics as an integral component of communication, which in some cases drives the interaction between the user and the robot, and we study its relation to speech and gestures. We illustrate features of interactions between an elderly person and an assistant via excerpts from our ongoing data collection. We also describe the architecture of our interface and ongoing research to bring this interface to fruition.

Future service robots applications in healthcare may require systems to be adaptable in terms of verbal and non-verbal behaviors to ensure patient perceptions of quality healthcare. Adaptation of robot behaviors should account for patient emotional states. Related to this, there is a need for a reliable method by which to classify patient emotions in real-time during patient-robot interaction (PRI). Accurate emotion classification could facilitate appropriate robot adaptation and effective healthcare operations (e.g., medicine delivery). We conducted and compared two simulated robot medicine delivery experiments with different participant age groups and robot configurations. A meta-analysis of the data from these experiments was to identify a robust approach for emotional state classification across age groups and robot configurations. Results revealed age differences as well as multiple robot humanoid feature manipulations to cause inaccuracy in emotion classification using statistical and machine learning methods. Younger adults tend to have higher emotional variability than elderly. Combinations of robot features were also found to induce emotional uncertainty and extreme responses. These findings were largely reflected in terms of physiological responses rather than subjective reports of emotions.

As robots become incorporated into our environments, they must be equipped with the ability to communicate effectively with us. In particular, robots that perform longer tasks for a small set of people (e.g., a companion robot to escort visitors to meetings all day) need to be able to start and maintain interesting and relevant dialog with any and all humans involved.In this work, we present our ongoing work on our robot, CoBot, which is assigned an all-day task to escort a visitor around our building and perform tasks for her. We first describe CoBot's dialog manager which is responsible for the task-oriented dialog, including dialog to meet the visitor's needs, CoBot's notifications of interesting locations around the building, and the robot's own requests for help. We, then, focus two aspects of the dialog manager: 1) how CoBot can invoke more accurate answers to its requests for help from the visitor and 2) how to reduce repetitive dialog which can happen during all-day interactions. We provide an example dialog between CoBot and a visitor to illustrate the dialog manager's capabilities.

We summarize our past, present and future research related to human-robot dialogue, starting with its foundations in collaborative discourse theory, continuing to our current research on recognizing and generating engagement, and concluding with an outline of new work we are beginning on the modeling of long-term relationships between humans and robots.

We are motivated by building a system for an autonomous robot companion that collaborates with a human partner for achieving a common mission. The objective of the robot is to infer the human's preferences upon the tasks of the mission so as to collaborate with the human by achieving human's non-favorite tasks. Inspired by recent researches about the recognition of human's intention, we propose a unified model that allows the robot to switch accurately between verbal and non-verbal interactions. Our system unifies an epistemic partially observable Markov decision process (POMDP) that is a human-robot spoken dialog system aiming at disambiguating the human's preferences and an intuitive human-robot collaboration consisting in inferring human's intention based on the observed human actions. The beliefs over human's preferences computed during the dialog are then reinforced in the course of the task execution by the intuitive interaction. Our unified model helps the robot inferring the human's preferences and deciding which tasks to perform to effectively satisfy these preferences. The robot is also able to adjust its plan rapidly in case of sudden changes in the human's preferences and to switch between both kind of interactions. Experimental results on a scenario inspired from robocup@home outline various specific behaviors of the robot during the collaborative mission.

As a precursor to learning to use language an infant has to acquire preliminary linguistic skills, including the ability to recognize and produce word forms without meaning. This develops out of babbling, through vocal interaction with carers. We report on evidence from developmental psychology and from neuroscientific research that supports a dual process approach to language learning. We describe a simulation of the transition from babbling to the recognition of first word forms in a simulated robot interacting with a human teacher. This precedes interactions with the real iCub robot.

This paper presents a robust methodology for grounding vocabulary in robots. A social language grounding experiment is designed, where, a human instructor teaches a robotic agent the names of the objects present in a visually shared environment. Any system for grounding vocabulary has to incorporate the properties of gradual evolution and lifelong learning. The learning model of the robot is adopted from an ongoing work on developing systems that conform to these properties. Significant modifications have been introduced to the adopted model, especially to handle words with multiple meanings. A novel classification strategy has been developed for improving the performance of each classifier for each learned category. A set of six new nearest-neighbor based classifiers have also been integrated into the agent architecture. A series of experiments were conducted to test the performance of the new model on vocabulary acquisition. The robot was shown to be robust at acquiring vocabulary and has the potential to learn a far greater number of words (with either single or multiple meanings).