REALM: Real-Time Estimates of Assistance for Learned Models in Human-Robot Interaction

Personal use of this material is permitted. ACCEPTED APRIL, 2025 1 REALM: Real-Time Estimates of Assistance for Learned Models in Human-Robot Interaction Michael Hagenow 1 and Julie A. Shah 1 Abstract --There are a variety of mechanisms (i.e., input types) for real-time human interaction that can facilitate effective human-robot teaming. For example, previous works have shown how teleoperation, corrective, and discrete (i.e., preference over a small number of choices) input can enable robots to complete complex tasks. However, few previous works have looked at combining different methods, and in particular, opportunities for a robot to estimate and elicit the most effective form of assistance given its understanding of a task. In this paper, we propose a method for estimating the value of different human assistance mechanisms based on the action uncertainty of a robot policy. Our key idea is to construct mathematical expressions for the expected post-interaction differential entropy (i.e., uncertainty) of a stochastic robot policy to compare the expected value of different interactions. As each type of human input imposes a different requirement for human involvement, we demonstrate how differential entropy estimates can be combined with a likelihood penalization approach to effectively balance feedback informational needs with the level of required input. We demonstrate evidence of how our approach interfaces with emergent learning models (e.g., a diffusion model) to produce accurate assistance value estimates through both simulation and a robot user study. Our user study results indicate that the proposed approach can enable task completion with minimal human feedback for uncertain robot behaviors. I NTRODUCTION F OR complex and critical tasks, it is beneficial to maintain a skilled human operator in the loop who can ensure appropriate task outcomes. Within the broad umbrella of human-in-the-loop (HIL) methods, there are many different levels of automation and corresponding mechanisms of human input; including traded teleoperation (i.e., alternating periods of teleoperation and autonomy), control of robot subspaces (e.g., the human controls only rotation or position), and discrete input. However, few works have explored methods where robots elicit different levels of human feedback in real-time during task execution.