Communicating with Executable Action Representations

Natural language instructions are often underspecified and imprecise which makes them hard to understand for an artificial agent. In this article we present a system of connected knowledge representations that is used to control a robot through instructions. As actions are a key component of instructions and the robot's behavior the representation of action is central in our approach. First, the system consists of a conceptual schema representation which provides a parameter interface for action. Second, we present an intermediate representation of the temporal structure of action and show how this generic action structure can be mapped to detailed action controllers as well as language.