Safe Multi-Agent Reinforcement Learning for Formation Control without Individual Reference Targets

Abstract--In recent years, formation control of unmanned vehicles has received considerable interest, driven by the progress in autonomous systems and the imperative for multiple vehicles to carry out diverse missions. In this paper, we address the problem of behavior-based formation control of mobile robots, where we use safe multi-agent reinforcement learning (MARL) to ensure the safety of the robots by eliminating all collisions during training and execution. To ensure safety, we implemented distributed model predictive control safety filters to override unsafe actions. We focus on achieving behavior-based formation without having individual reference targets for the robots, and instead use targets for the centroid of the formation. This formulation facilitates the deployment of formation control on real robots and improves the scalability of our approach to Figure 1: Real-world example for behavior-based formation control of more robots. The task cannot be addressed through optimizationbased mobile robots based on centroid reference targets. The formation is controllers without specific individual reference targets for defined by the distances between the three robots. The robots start the robots and information about the relative locations of each from random locations and then navigate cooperatively to move the robot to the others. That is why, for our formulation we use target for the centroid of the formation while aiming to maintain the MARL to train the robots. Moreover, in order to account for the predefined distances with respect to each other. The centroid of the interactions between the agents, we use attention-based critics to formation reaches the first goal and is then moved to the second goal improve the training process.