The AI Debate Critical To The Future Of Autonomous Vehicles

This is because deep-learning neural networks are a "black box": they consist of millions of connections between nodes that are fine-tuned in opaque, subtle ways as data is fed in. When a deep-learning network produces an output (e.g., the decision to stop or not to stop at a yellow light), that output cannot be traced to a particular sequence in the AV's software; rather, it is an emergent outcome of the overall system. Experts call this problem "lack of interpretability." As well as deep learning networks may perform at driving 99.9% of the time, this lack of interpretability becomes a real concern on those rare occasions when an AV makes the wrong decision and causes an accident. In those situations, humans have no way to explain what went wrong and no way to troubleshoot the error.