Probabilistic Perspectives on Error Minimization in Adversarial Reinforcement Learning