Visual Diagnostics for Deep Reinforcement Learning Policy Development

Abstract--Modern vision-based reinforcement learning techniques often use convolutional neural networks (CNN) as universal function approximators to choose which action to take for a given visual input. Until recently, CNNs have been treated like black-box functions, but this mindset is especially dangerous when used for control in safety-critical settings. We use a simulated drone environment as an example scenario. These visualization algorithms are an important tool for behavior introspection and provide insight into the qualities and flaws of trained policies when interacting with the physical world. A video may be seen at https://sites.google.com/view/drlvisual. Einforcement learning (RL) is a family of methods aimed at training an agent to collect rewards from an environment through trial-and-error approaches.