Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing

Although we have focused on the "hot" area of A Vs, our discussion and the novel CBI theorems are more generally applicable. We see them as especially useful now for MLbased systems with critical applications, although not with extreme requirements, since assurance in these systems must rely on combinations of statistical evidence with other verification methods that are, as yet, not well-established. A PPENDIX A. Statement And Proof of CBI Theorem 1 Problem: Consider the set D of all probability distributions defined over the unit interval, each distribution representing a potential prior distribution of pfm values for an A V . For 0 p l null null 1, we seek a prior distribution that minimises the posterior confidence in a reliability bound p [ p l, 1], given k fatalities have occurred over n miles driven and subject to constraints on some quantiles of the prior distribution. That is, for θ (0, 1], we solve minimise D Pr ( X null p k & n) subject to Pr ( X null null) θ, Pr (X null p l) 1 Solution: There is a prior in D that minimises the posterior confidence: the 2-point distribution Pr ( X x) θ 1 x x 1 (1 θ)1 x x 3 where p l null x 1 null null x 3, and the values of x 1 and x 3 both depend on the model parameters (i.e.