Note: This manuscript has been accepted for publication at a conference in 2024 and will be published under the same title. The version in this pre-print will undergo minor edits and thus does not represent the final version of this work. Abstract-- Owing to the recent success of Large Language Models, Modern A.I has been much focused on linguistic interactions with humans but less focused on nonlinguistic forms of communication between man and machine. In the present paper, we test how affective-linguistic communication, in combination with differential outcomes training, affects mutual learning in a human-robot context. Taking inspiration from child-caregiver dynamics, our human-robot interaction setup consists of a (simulated) robot attempting to learn how best to communicate internal, homeostatically-controlled needs; while a human "caregiver" attempts to learn the correct object to satisfy the robot's present communicated need. We studied the effects of i) human training type, and ii) robot reinforcement learning type, to assess mutual learning terminal accuracy and rate of learning (as measured by the average reward achieved by the robot). Our results find mutual learning between a human and a robot is significantly improved with Differential Outcomes Training (DOT) compared to Non-DOT (control) conditions. We find further improvements when the robot uses an exploration-exploitation policy selection, compared to purely exploitation policy selection. These findings have implications for utilizing socially assistive robots (SAR) in therapeutic contexts, e.g. for cognitive interventions, and educational applications.

Reinforcement learning models address animal's behavioral adaptation to its changing "external" environment, and are based on the assumption that Pavlovian, habitual and goal-directed responses seek to maximize reward acquisition. Negative-feedback models of homeostatic regulation, on the other hand, are concerned with behavioral adaptation in response to the "internal" state of the animal, and assume that animals' behavioral objective is to minimize deviations of some key physiological variables from their hypothetical setpoints. Building upon the drive-reduction theory of reward, we propose a new analytical framework that integrates learning and regulatory systems, such that the two seemingly unrelated objectives of reward maximization and physiological-stability prove to be identical. The proposed theory shows behavioral adaptation to both internal and external states in a disciplined way. We further show that the proposed framework allows for a unified explanation of some behavioral pattern like motivational sensitivity of different associative learning mechanism, anticipatory responses, interaction among competing motivational systems, and risk aversion.

Obesity is a critical healthcare issue affecting the United States. The least risky treatments available for obesity are behavioral interventions meant to promote diet and exercise. Often these interventions contain a mobile component that allows interventionists to collect participants level data and provide participants with incentives and goals to promote long term behavioral change. Recently, there has been interest in using direct financial incentives to promote behavior change. However, adherence is challenging in these interventions, as each participant will react differently to different incentive structure and amounts, leading researchers to consider personalized interventions. The key challenge for personalization, is that the clinicians do not know a priori how best to administer incentives to participants, and given finite intervention budgets how to disburse costly resources efficiently. In this paper, we consider this challenge of designing personalized weight loss interventions that use direct financial incentives to motivate weight loss while remaining within a budget. We create a machine learning approach that is able to predict how individuals may react to different incentive schedules within the context of a behavioral intervention. We use this predictive model in an adaptive framework that over the course of the intervention computes what incentives to disburse to participants and remain within the study budget. We provide both theoretical guarantees for our modeling and optimization approaches as well as demonstrate their performance in a simulated weight loss study. Our results highlight the cost efficiency and effectiveness of our personalized intervention design for weight loss.

Motivation impacts people's lives in a powerful way and is at the heart of a plethora of day-to-day activities and achievement settings, from success at the workplace to learning and acquiring knowledge to trying to quit bad habits. The current work aims to develop an adaptive robot companion that models a user's daily motivational state and chooses appropriate motivational strategies to keep the user on track for achieving a daily goal. The two main components we are focusing on in this context are creating an ontology-based user model of the person's motivational states and using an appropriate strategy selection algorithm that chooses the best motivational strategies for the user each day based on the user model's output. Specifically, we are focusing on the important application domain of physical activity and aim to help early adolescents achieve daily-recommended levels of physical activity. Our human-robot interaction system uses information acquired from the user to feed the user model and physical activity data from a wristband device to inform the strategy selection algorithm.

Last month at the San Francisco Museum of Modern Art I saw "2001: A Space Odyssey" on the big screen for my 47th time. The fact that this masterpiece remains on nearly every relevant list of "top ten films" and is shown and discussed over a half-century after its 1968 release is a testament to the cultural achievement of its director Stanley Kubrick, writer Arthur C. Clarke, and their team of expert filmmakers. As with each viewing, I discovered or appreciated new details. But three iconic scenes -- HAL's silent murder of astronaut Frank Poole in the vacuum of outer space, HAL's silent medical murder of the three hibernating crewmen, and the poignant sorrowful "death" of HAL -- prompted deeper reflection, this time about the ethical conundrums of murder by a machine and of a machine. In the past few years experimental autonomous cars have led to the death of pedestrians and passengers alike. AI-powered bots, meanwhile, are infecting networks and influencing national elections. Elon Musk, Stephen Hawking, Sam Harris, and many other leading AI researchers have sounded the alarm: Unchecked, they say, AI may progress beyond our control and pose significant dangers to society. When astronauts Frank and Dave retreat to a pod to discuss HAL's apparent malfunctions and whether they should disconnect him, Dave imagines HAL's views and says: "Well I don't know what he'd think about it."

`No' belongs to the first ten words used by children and embodies the first active form of linguistic negation. Despite its early occurrence the details of its acquisition process remain largely unknown. The circumstance that `no' cannot be construed as a label for perceptible objects or events puts it outside of the scope of most modern accounts of language acquisition. Moreover, most symbol grounding architectures will struggle to ground the word due to its non-referential character. In an experimental study involving the child-like humanoid robot iCub that was designed to illuminate the acquisition process of negation words, the robot is deployed in several rounds of speech-wise unconstrained interaction with na\"ive participants acting as its language teachers. The results corroborate the hypothesis that affect or volition plays a pivotal role in the socially distributed acquisition process. Negation words are prosodically salient within prohibitive utterances and negative intent interpretations such that they can be easily isolated from the teacher's speech signal. These words subsequently may be grounded in negative affective states. However, observations of the nature of prohibitive acts and the temporal relationships between its linguistic and extra-linguistic components raise serious questions over the suitability of Hebbian-type algorithms for language grounding.

Motivation impacts people’s lives in a powerful way and is at the heart of a plethora of day-to-day activities and achievement settings, from success at the workplace to learning and acquiring knowledge to trying to quit bad habits. The current work aims to develop an adaptive robot companion that models a user’s daily motivational state and chooses appropriate motivational strategies to keep the user on track for achieving a daily goal. The two main components we are focusing on in this context are creating an ontology-based user model of the person’s motivational states and using an appropriate strategy selection algorithm that chooses the best motivational strategies for the user each day based on the user model’s output. Specifically, we are focusing on the important application domain of physical activity and aim to help early adolescents achieve daily-recommended levels of physical activity. Our human-robot interaction system uses information acquired from the user to feed the user model and physical activity data from a wristband device to inform the strategy selection algorithm.

Reinforcement learning models address animal's behavioral adaptation to its changing "external" environment, and are based on the assumption that Pavlovian, habitual and goal-directed responses seek to maximize reward acquisition. Negative-feedback models of homeostatic regulation, on the other hand, are concerned with behavioral adaptation in response to the "internal" state of the animal, and assume that animals' behavioral objective is to minimize deviations of some key physiological variables from their hypothetical setpoints. Building upon the drive-reduction theory of reward, we propose a new analytical framework that integrates learning and regulatory systems, such that the two seemingly unrelated objectives of reward maximization and physiological-stability prove to be identical. The proposed theory shows behavioral adaptation to both internal and external states in a disciplined way. We further show that the proposed framework allows for a unified explanation of some behavioral phenomenon like motivational sensitivity of different associative learning mechanism, anticipatory responses, interaction among competing motivational systems, and risk aversion.

This study investigates Serious Games (SG) to assess motivational factors appropriate to an Intelligent Tutoring System (ITS). An ITS can benefit from SG’ elements that can highly support learners’ motivation. Thus, identifying and assessing the effect that these factors may have on learners is a crucial step before attempting to integrate them into an ITS. We designed an experiment using a Serious Game and combined both the theoretical ARCS model of motivation and empirical physiological sensors (heart rate, skin conductance and EEG) to assess the effects of motivational factors on learners. We then identified physiological patterns correlated with one motivational factor in a Serious Game (Alarm triggers) associated with the Attention category of the ARCS model. The best result of three classifiers run on the physiological data has reached an accuracy of 73.8% in identifying learners’ attention level as being either above or below average. These results open the door to the possibility for an ITS to discriminate between attentive and inattentive learners.

The past decade has witnessed an increasing interest in the use of virtual coaches in healthcare. This paper describes a virtual coach to provide mindfulness meditation training, and the coaching support necessary to begin a regular practice. The coach is implemented as an embodied conversational character, and provides mindfulness training and coaching support via a web-based application. The coach is represented as a female character, capable of showing a variety of affective and conversational expressions, and interacts with the user via a mixed-initiative, text-based, natural language dialogue. The coach adapts both its facial expressions and the dialogue content to the user’s learning needs and motivational state. Findings from a pilot evaluation study indicate that the coach-based training is more effective in helping users establish a regular practice than self-administered training via written and audio materials. The paper concludes with an analysis of the coach features that contribute to these results, discussion of key challenges in affect-adaptive coaching, and plans for future work.