Using Cognitive Models to Train Warm Start Reinforcement Learning Agents for Human-Computer Interactions

Reinforcement learning (RL) has gained growing popularity in many human-computer interaction (HCI) applications [1, 2, 3]. In digital health interventions, for example, RL is a natural choice for personalization as RL agents can continuous adapt their strategies based on users' responses to the interventions [3]. Moreover, the recent advances in interactive RL calls for contributions from HCI researchers to improve the efficiency of RL algorithms [4]. While there is a natural fit between RL and HCI, the well-known data greedy property of reinforcement learning makes the RL-based systems often suffer from the cold start problem [5]. In HCI, as very few (or even no) experiences with users are available at the beginning in general, RL agents are required to interact many times with users prior to performing well. Many researchers had made efforts to overcome this challenge by shortening the learning process. Several approaches have been proposed to perform a faster online learning so that less interactions are needed in practice. For instance, Tabatabaei et al. [6] and Tomkins et al. [7] make RL algorithms quickly learn from the limited experience This is a preprint of our position paper presented to the "Reinforcement Learning for Humans, Computer, and Interaction (RL4HCI)" workship at ACM CHI2021, https://sites.google.com/view/rl4hci/home. The preprint is published under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.