Formulation (2) contains several interesting examples in machine learning such as robust optimization [14, 46] and adversarial training [17, 40].

Exploration is a key problem in reinforcement learning, since agents can only learn from data they acquire in the environment.