Safe Control Transitions: Machine Vision Based Observable Readiness Index and Data-Driven Takeover Time Prediction

University of California San Diego USA Pujitha Gunaratne Toyota Collaborative Safety Research Center USA Paper Number 23-0331 ABSTRACT To make safe transitions from autonomous to manual control, a vehicle must have a representation of the awareness of driver state; two metrics which quantify this state are the Observable Readiness Index and Takeover Time. In this work, we show that machine learning models which predict these two metrics are robust to multiple camera views, expanding from the limited view angles in prior research. Importantly, these models take as input feature vectors corresponding to hand location and activity as well as gaze location, and we explore the tradeoffs of different views in generating these feature vectors. Further, we introduce two metrics to evaluate the quality of control transitions following the takeover event (the maximal lateral deviation and velocity deviation) and compute correlations of these post-takeover metrics to the pre-takeover predictive metrics. INTRODUCTION It is important to plan for safe operation of intelligent vehicles in situations of system failure. Intelligent and autonomous vehicles face challenges when dealing with long-tail events, defined as events which occur with little to no regularity and are thus difficult for the dominant regime of learning-based perception and control models to operate safely. When such situations are identified, the vehicle may benefit from passing control to the human driver.