How to Plan When Being Deliberately Misled

Reasoning agents are often faced with the need to robustly deal with erroneous information. When a robot given the task of returning with the red cup from the kitchen table arrives in the kitchen to find no red cup but instead notices a blue cup and a red plate on the table, what should it do? The best course of action is to attempt to salvage the situation by relying on its preferences to return with one of the objects available. We provide a solution to this problem using the Situation Calculus extended with a notion of belief. We then provide an efficient practical implementation by mapping this formalism into default rules for which we have an implemented solver.