Robust Reinforcement Learning via Adversarial Kernel Approximation

In reinforcement learning (RL), we are concerned with learning good policies for sequential decisionmaking problems modeled as Markov Decision Processes (MDPs) [29, 35]. MDPs assume that the transition model of the environment is fixed across training and testing, but this is often violated in practical applications. For example, when deploying a simulator-trained robot in reality, a notable challenge is the substantial disparity between the simulated environment and the intricate complexities of the real world, leading to potential subpar performance upon deployment. Such a mismatch may significantly degrade the performance of the trained policy in testing. To deal with this issue, the robust MDP (RMDP) framework has been introduced in [16, 24, 44], aiming to learn policies that are robust to perturbation of the transition model within an uncertainty set.