Beelines: Evaluating Motion Prediction Impact on Self-Driving Safety and Comfort

Abstract-- The commonly used metrics for motion prediction do not correlate well with a self-driving vehicle's systemlevel Since high-fidelity simulations and track testing can be resource-intensive, the use of prediction metrics better correlated with full-system behavior allows for swifter (a) The future prediction (blue) is in the vehicle's path iteration cycles. In this paper, we offer a conceptual framework while the future ground-truth (orange) is not. These models Figure 1: Two examples of predictions errors, with identical are usually trained on variants of the L2 or cross-entropy displacement error. The bottom is more of a safety concern. However, consider the shortcoming illustrated by Figure 1, which shows This work proposes a metrics framework that directly two simple scenarios with identical displacement error, but in connects precision and recall to the self-driving specific which the error in Figure 1b is more severe because the vehicle concerns of ride comfort and safety.