On the Limit of Language Models as Planning Formalizers