Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guarantees

Therefore, we require methods that can guarantee robust and safe performance under general forms of environment uncertainty. Robustness and safety are critical for the trustworthy Unfortunately, popular approaches to robustness deployment of deep reinforcement learning in deep RL consider very structured forms of uncertainty in real-world decision making applications. In in order to facilitate efficient implementations. Adversarial particular, we require algorithms that can guarantee methods implement a specific type of perturbation, such robust, safe performance in the presence of as the application of a physical force (Pinto et al., 2017) general environment disturbances, while making or a change in the action that is deployed (Tessler et al., limited assumptions on the data collection process 2019a). Parametric approaches, on the other hand, consider during training. In this work, we propose a safe reinforcement robustness with respect to environment characteristics that learning framework with robustness can be altered in a simulator (Rajeswaran et al., 2017; Peng guarantees through the use of an optimal transport et al., 2018; Mankowitz et al., 2020). When we lack domain cost uncertainty set. We provide an efficient, knowledge on the structure of potential disturbances, these theoretically supported implementation based on techniques may not guarantee robustness and safety. Optimal Transport Perturbations, which can be applied in a completely offline fashion using only Another drawback of existing approaches is their need to data collected in a nominal training environment.