Extracting interesting scenarios from real-world data as well as generating failure cases is important for the development and testing of autonomous systems. We propose efficient mechanisms to both characterize and generate testing scenarios using a state-of-the-art driving simulator. For any scenario, our method generates a set of possible driving paths and identifies all the possible safe driving trajectories that can be taken starting at different times, to compute metrics that quantify the complexity of the scenario. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project, as well as adversarial scenarios generated in simulation. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident. We demonstrate a strong correlation between the proposed metrics and human intuition.

Items (a), (b), and (c) are integrated into a intelligence (AI) and machine learning (ML) to integrate Bayesian network (BN). BNs provide a favorable formalism mechanical, electronic, and computing technologies to make in which to model the propagation of faults across AV system real-time driving decisions. AI enables AVs to navigate through components with an interpretable model. The model, together complex environments while maintaining a safety envelope [1], with fault injection results, can be used to design and assess [2] that is continuously measured and quantified by onboard the safety of AVs. Further, BNs enable rapid probabilistic sensors (e.g., camera, LiDAR, RADAR) [3]-[5]. Clearly, the inference, which allows DriveFI to quickly find safety-critical safety and resilience of AVs are of significant concern, as faults. The Bayesian FI framework can be extended to other exemplified by several headline-making AV crashes [6], [7], safety-critical systems (e.g., surgical robots). The framework as well as prior work characterizing AV resilience during road requires specification of the safety constraints and the system tests [8]. Hence there is a compelling need for a comprehensive software architecture to model causal relationship between assessment of AV technology.