For robots to effectively interact with human users, it is necessary that they recognize what people in the environment are doing. This is especially the case when robots are performing complementary tasks since the human users are not following any specific process. There is much uncertainty in how people act and the duration of time they need to perform their actions. In this work, we discuss the use of topic models for such plan and activity recognition tasks. We begin with the development of a domain-independent representation of human postural information obtained from RGB-D sensor data. This representation may be used with Latent Dirichlet Allocation (LDA) topic models as an integration of plan and activity recognition. This is followed by a proposition of extensions to LDA that allow temporal and object relational information to also be used in plan and activity recognition tasks.

We examine new ways to perform plan recognition (PR) using natural language processing (NLP) techniques. PR often focuses on the structural relationships between consecutive observations and ordered activities that comprise plans. However, NLP commonly treats text as a bag-of-words, omitting such structural relationships and using topic models to break down the distribution of concepts discussed in documents. In this paper, we examine an analogous treatment of plans as distributions of activities. We explore the application of Latent Dirichlet Allocation topic models to human skeletal data of plan execution traces obtained from a RGB-D sensor. This investigation focuses on representing the data as text and interpreting learned activities as a form of activity recognition (AR). Additionally, we explain how the system may perform PR. The initial empirical results suggest that such NLP methods can be useful in complex PR and AR tasks.