Reviews: A Regularized Approach to Sparse Optimal Policy in Reinforcement Learning

Although some techniques are analogous to previous work (which is not bad per se, as it allows to apply more general regularisers within previous frameworks such as soft-actor-critic with small changes only), this work differs significantly from previous work and yields new insights how to obtain sparse policies or not. Claims are supported by proofs and experiments confirm that considering more flexible regularizations can be beneficial in different tasks. There are some issues with the continuous time case, see the section on improvements for details. Further the authors claim that trigonometric and exponential functions families yield multimodal policies (line 287). However, it is not clear to me how this is different to say entropy regularisation, and why a softmax policy cannot have multiple modes (unless of course I parameterize the policy with a single Gaussian in the continuous case, but this is a different issue).