Value of Information-based Deceptive Path Planning Under Adversarial Interventions

V alue of Information-based Deceptive Path Planning Under Adversarial Interventions Wesley A. Suttle, Jesse Milzman, Mustafa O. Karabag, Brian M. Sadler, Ufuk Topcu Abstract -- Existing methods for deceptive path planning (DPP) address the problem of designing paths that conceal their true goal from a passive, external observer . Such methods do not apply to problems where the observer has the ability to perform adversarial interventions to impede the path planning agent. In this paper, we propose a novel Markov decision process (MDP)-based model for the DPP problem under adversarial interventions and develop new value of information (V oI) objectives to guide the design of DPP policies. Using the V oI objectives we propose, path planning agents deceive the adversarial observer into choosing suboptimal interventions by selecting trajectories that are of low informational value to the observer . Leveraging connections to the linear programming theory for MDPs, we derive computationally efficient solution methods for synthesizing policies for performing DPP under adversarial interventions. In our experiments, we illustrate the effectiveness of the proposed solution method in achieving deceptiveness under adversarial interventions and demonstrate the superior performance of our approach to both existing DPP methods and conservative path planning approaches on illustrative gridworld problems. I NTRODUCTION Deceptive path planning (DPP) is the problem of designing a path that conceals its true objective from an outside observer. Several approaches to this problem have recently been developed, using model-based planning [1], [2], [3], [4] and model-free reinforcement learning [5], [6], [7], [8]. These methods make the strong assumption that the observer is passive and unable to affect the path planning agent's environment, however, significantly limiting their applicability.