Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Model

-- It is expected that many human drivers will still prefer to drive themselves even if the self-driving technologies are ready. T o enable A Vs to safely and efficiently maneuver in this mixed traffic, it is critical that the A Vs can understand how humans cope with risks and make driving-related decisions. On the other hand, the driving environment is highly dynamic and ever-changing, and it is thus difficult to enumerate all the scenarios and hard-code the controllers. T o face up these challenges, in this work, we incorporate a human decision-making model in reinforcement learning to control A Vs for safe and efficient operations. Specifically, we adapt regret theory to describe a human driver's lane-changing behavior, and fit the personalized models to individual drivers for predicting their lane-changing decisions. The predicted decisions are incorporated in the safety constraints for reinforcement learning in training and in implementation. We then use an extended version of double deep Q-network (DDQN) to train our A V controller within the safety set. By doing so, the amount of collisions in training is reduced to zero, while the training accuracy is not impinged. I. INTRODUCTION Autonomous driving has attracted significant research interest in the past two decades as it offers the potential to release drivers from exhausting driving. While great progresses have been made in the field of perception, path planning, and controls, high-level decision-making remains a big challenge due to the involvement of complex, cluttered environment and the dynamic, uncertain behaviors of other traffic users. Some recent works have been applying reinforcement learning (RL) methods to autonomous driving and promising performance [1] has been reported. RL-based methods can learn the decision-making and driving behaviors which are hard, if not infeasible, for traditional rule-based designs, and often with much less human effort. However, it is reported in [2] that when using RL-based methods lots of collisions happen before the agent starts to behave properly.