User Modeling Challenges in Interactive AI Assistant Systems

Interactive Artificial Intelligent(AI) assistant systems are designed to offer timely guidance to help human users to complete a variety tasks. One of the remaining challenges is to understand user's mental states during the task for more personalized guidance. In this work, we analyze users' mental states during task executions and investigate the capabilities and challenges for large language models to interpret user profiles for more personalized user guidance. In the digital age, there is immense potential for artificial intelligent (AI) assistant to guides users through complex tasks, from changing laptop batteries to piping frosting on a cake. One of the main challenges, however, lies in creating an interactive system that can not only understand which step the user is at, but can also detect user's mental states, such as frustration, familiarity with the task, detail-orientation, etc.