Infrastructure-based End-to-End Learning and Prevention of Driver Failure