Reviews: A Geometric Perspective on Optimal Representations for Reinforcement Learning

This paper studies the problem of learning useful representations for reinforcement learning through the lens of an adversarial framework. In particular, a good representation is identified as one that yields low linear value-function estimation error if an adversary is able to choose a value function (induced by a policy). The paper shows first that the the only policies that should be considered are deterministic, and then identifies a more narrowed set of adversarial values, though the number is still exponential. I really liked the theoretical insights of this paper, and because of this I tend to vote for acceptance, though I claim that experiments are too preliminary. Some more comments below: 1- in (1) highlight more clearly that \phi is the only optimization knob.