Choices are More Important than Efforts: LLM Enables Efficient Multi-Agent Exploration