Challenging Common Assumptions in Convex Reinforcement Learning