Review for NeurIPS paper: Variational Policy Gradient Method for Reinforcement Learning with General Utilities

Additional Feedback: Update: thanks for the answer, it helped clarify some points. I think the proposed additions will improve the clarity of the paper. While providing a common theoretical ground for general utilities in RL is not a minor contribution by any means, I would have loved to find a discussion on how to build upon these results. Do authors think their work can be leveraged to develop more efficient algorithm in the context of RL with general utilities, or the intended outcome is a deeper understanding of the setting without particular practical upsides? 2. Where the Variational Policy Gradient approach stands in comparison with other policy optimization methods for (specific) general utilities, e.g.