HMCF: A Human-in-the-loop Multi-Robot Collaboration Framework Based on Large Language Models

HMCF: A Human-in-the-loop Multi-Robot Collaboration Framework Based on Large Language Models Zhaoxing Li, Wenbo Wu, Y ue Wang, Y anran Xu, William Hunt, and Sebastian Stein Abstract -- Rapid advancements in artificial intelligence (AI) have enabled robots to perform complex tasks autonomously with increasing precision. Traditional approaches often lack generalization, requiring extensive engineering for new tasks and scenarios, and struggle with managing diverse robots. T o overcome these limitations, we propose a Human-in-the-loop Multi-Robot Collaboration Framework (HMCF) powered by large language models (LLMs). LLMs enhance adaptability by reasoning over diverse tasks and robot capabilities, while human oversight ensures safety and reliability, intervening only when necessary. Each robot is equipped with an LLM agent capable of understanding its capabilities, converting tasks into executable instructions, and reducing hallucinations through task verification and human supervision. Simulation results show that our framework outperforms state-of-the-art task planning methods, achieving higher task success rates with an improvement of 4.76%. Real-world tests demonstrate its robust zero-shot generalization feature and ability to handle diverse tasks and environments with minimal human intervention. I NTRODUCTION The rapid progress of artificial intelligence (AI) and robotics has significantly advanced multi-robot systems (MRSs), enabling them to perform increasingly complex tasks with high autonomy and precision [1]. These systems are deployed in various real-world applications, including disaster response, industrial automation, logistics, and healthcare [2], [3]. By coordinating multiple robots, MRSs can efficiently handle large-scale tasks that exceed the capabilities of individual robots.