Reviews: Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints

This paper formalizes the problem of inverse reinforcement learning in which the learner's goal is not only to imitate the teacher's demonstration, but also to satisfy her own preferences and constraints. It analyzes the suboptimality of learner-agnostic teaching, where the teacher gives demonstrations without considering the learner's preferences. It then proposes a learner-aware teaching algorithm, where the teacher selects demonstrations while accounting for the learner's preferences. It considers different types of learner models with hard or soft preference constraints. It also develops learner-aware teaching methods for both cases where the teacher has full knowledge of the learner's constraints or does not know it.