Is RLHF More Difficult than Standard RL? A Theoretical Perspective