LLM Granularity for On-the-Fly Robot Control