Multi-vehicle Conflict Resolution in Highly Constrained Spaces by Merging Optimal Control and Reinforcement Learning

Abstract: We present a novel method to address the problem of multi-vehicle conflict resolution in highly constrained spaces. An optimal control problem is formulated to incorporate nonlinear, non-holonomic vehicle dynamics and exact collision avoidance constraints. A solution to the problem can be obtained by first learning configuration strategies with reinforcement learning (RL) in a simplified discrete environment, and then using these strategies to shape the constraint space of the original problem. Simulation results show that our method can explore efficient actions to resolve conflicts in confined space and generate dexterous maneuvers that are both collision-free and kinematically feasible. Keywords: Trajectory and Path Planning, Multi-vehicle systems, Autonomous Vehicles, Reinforcement learning control, Control problems under conflict 1. INTRODUCTION When conflicts arise in highly constrained spaces such as crowded parking lots, both the optimal control and the RL approaches often fail due to the following reasons: Current autonomous vehicles (AVs) operate reasonably well in environments where traffic rules are well-defined, (i) The vehicles need to plan for combinatorial actions in the surrounding agents are rational, and their actions can order to create spaces for each other to pass through; be easily predicted.