Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents