We consider the problem of providing users of deep Reinforcement Learning (RL) based systems with a better understanding of when their output can be trusted. We offer an explainable artificial intelligence (XAI) framework that provides a three-fold explanation: a graphical depiction of the systems generalization and performance in the current game state, how well the agent would play in semantically similar environments, and a narrative explanation of what the graphical information implies. We created a user-interface for our XAI framework and evaluated its efficacy via a human-user experiment. The results demonstrate a statistically significant increase in user trust and acceptance of the AI system with explanation, versus the AI system without explanation.

Control of the air in envisioned large-scale battles against near-peer adversaries will require revolutionary new approaches to airborne mission management, where decision authority and platform autonomy are dynamically delegated and functional roles and combat capabilities are assigned across multiple distributed tiers of platforms and human operators. System capabilities range from traditional airborne battle managers, to manned tactical aviators, to autonomous unmanned aerial systems. Due to the overwhelming complexity, human operators will require the assistance of advanced autonomy decision aids with new mechanisms for operator supervision and management of teams of manned and unmanned systems. In this paper we describe a conceptual distributed mission management approach that employs novel human-automation teaming constructs to address the complexity of envisioned operations in highly contested environments. We then discuss a cognitive engineering approach to designing roleand task-tailored human machine interfaces between humans and the autonomous systems. We conclude with a discussion of multi-level evaluation approaches for experimentation.