Revisiting Generative Policies: A Simpler Reinforcement Learning Algorithmic Perspective