Large Language Models Are Semi-Parametric Reinforcement Learning Agents