Reviews: Reinforcement Learning with Convex Constraints

The paper presents a way to solve the approachibility problem in RL by reduction to a standard RL problem. It casts this problem as a zero-sum game using conic duality, which is solved by a primal-dual technique based on tools from online learning. The proposed algorithm assumes an oracle that approximately solves a standard RL problem. It runs primal-dual iterations, where the dual part of the algorithm updates measurement weights according to the current primal solution obtained from the oracle. Originality: This work introduces a new problem of finding policy those measurements vectors lies inside a convex target set.