Query-Efficient Planning with Language Models