We apply diffusion strategies to develop a fully-distributed cooperative reinforcement learning algorithm in which agents in a network communicate only with their immediate neighbors to improve predictions about their environment. The algorithm can also be applied to off-policy learning, meaning that the agents can predict the response to a behavior different from the actual policies they are following. The proposed distributed strategy is efficient, with linear complexity in both computation time and memory footprint. We provide a mean-square-error performance analysis and establish convergence under constant step-size updates, which endow the network with continuous learning capabilities. The results show a clear gain from cooperation: when the individual agents can estimate the solution, cooperation increases stability and reduces bias and variance of the prediction error; but, more importantly, the network is able to approach the optimal solution even when none of the individual agents can (e.g., when the individual behavior policies restrict each agent to sample a small portion of the state space).

In this paper, we translate a model of value-based emo- tional agents into an architecture for narrative characters and we validate it in a narrative scenario. The advantage of using such model is that different moral behaviors can be obtained as a consequence of the emotional ap- praisal of moral values, a desirable feature for digital storytelling techniques.

AI research in interactive narrative often lacks specificity as to the player experience it is trying to enable. In this paper, we consider a set of desirable elements from narrative and interactive experiences, and show by looking at playable experiences from industry and academia that combining them has the potential to be limited or self-defeating. To address these issues, we propose opportunistic storytelling , a set of design principles for near-term playable interactive narratives.

As human-robot teamwork becomes increasingly common, a key challenge is to fluidly and intuitively coordinate team members' interactions. In this work, we explore two modalities of human-robot coordination: active, where agents intentionally attempt to understand and influence the plans of human teammates, and passive, where agents simply react to their human teammates' varying behavior. In our Productivity and Wellness Pal (PaWPal) project, we seek to develop an agent that actively elicits a teammate's constraints, preferences, and goals in order to nudge them towards better behavior. Conversely, in our Coordinating Human-Robot Teamwork project, we take a distributed approach to scheduling where agents passively adapt to teammates' plan executions. Our research hypothesis is that human-robot coordination Figure 1: Screenshots from our ESM study, conducted using techniques will lead to more natural and effective PACO on Android (http://www.pacoapp.com/).

The lure of understanding biological intelligence has long occupied researchers. Success has always been measured in peer review, number of citations, or how influential some piece of work is in inspiring the next generation of re- searchers. What human-robot interaction (HRI) and artificial intelligence (AI) promises is a metric of believability that is not intrinsic to the values of the researcher or community of practice but to the utility and successful function of the robotic artifact within a larger society. This paper is a reflec- tion and response to the hypothesis that HRI is a pure, funda- mental art of artificial intelligence and the last great successor to a domain fraught with the trappings of an art that lost its way.

In this work, we focus on advancing the state of the art in intelligent agents that can learn complex procedural tasks from humans. Our main innovation is to view the interaction between the human and the robot as a mixed- initiative collaboration. Our contribution is to integrate hierarchical task networks and collaborative discourse theory into the learning from demonstration paradigm to enable robots to learn complex tasks in collaboration with the human teacher.

Vocalics is the study of the nonverbal aspects of speech, such as volume, pitch, and rate. Our contribution is a parametric We present an overview of the control, recognition, decision-making, vocalic behavior controller that autonomously adjusts and learning techniques utilized by the Interaction the robot speaker volume based on models of how a Lab (robotics.usc.edu/interaction) at the University human user will hear speech produced by the robot. These of Southern California (USC) to enable autonomy in sociable models vary with distance, orientation, and perceived environmental and socially assistive robots. These techniques are implemented interference (Mead & Matarić 2014). Our future with two software libraries: 1) the Social Behavior work will investigate adapting the pitch and rate of speech Library (SBL) provides autonomous social behavior produced by a robot to improve user speech perception.

CoBot is a service mobile robot that has been continuously Frame: GoTo deployed for extended periods of time in a multi-floor - Parameters: destination office-style building (Biswas and Veloso 2013). While moving in the building CoBot, is able to perform multiple tasks Frame: DeliverObject for its users; the robot is able to autonomously navigate to - Parameters: object, source, destination any of the rooms in the building, to deliver objects and messages and to escort visitors to their destination. While apparently Figure 1: Semantic frames of the two task CoBot is able to very different, all the tasks CoBot is able to perform execute from spoken commands. Often, only being able to move, is not enough to accomplish the task required; if CoBot needs to deliver an object, given that it does not have arms, it cannot pick it up by itself, similarly when it needs to travel across floors it cannot push the elevator button. In order to overcome its limitation CoBot ask for help to humans, either the user or bypasser, achieving symbiotic autonomy (Rosenthal, Biswas, and Veloso 2010).

Over the last five years, and while developing an architecture for autonomous service robots in human environments, we have identified several key decisional issues that are to be tackled for a cognitive robot to share space and tasks with a human. We introduce some of them here: situation assessment and mutual modelling, management and exploitation of each agent (human and robot) knowledge in separate cognitive models, natural multi-modal communication, "human-aware" task planning, and human and robot interleaved plan achievement. As a general "take home" message, it appears that explicit knowledge management, both symbolic and geometric, proves to be a successful key while attempting to address these challenges, as it pushes for a different, more semantic way to address the decision-making issue in human-robot interactions.

We are looking at new ways of building algorithms for synthesizing and rendering animation in social robots that can keep them as interactive as necessary, while still following on principles and practices used by professional animators. We will be studying the animation process side by side with professional animators in order to understand how these algorithms and tools can be used by animators to achieve animation capable of correctly adapting to the environment and the artificial intelligence that controls the robot. Figure 1: Two example scenarios featuring a touch-based Robotic characters are becoming widespread as useful multimedia application, sensors, and different robots.