Large Language Models as Agents in the Clinic