Can language agents be alternatives to PPO? A Preliminary Empirical Study On OpenAI Gym