Is semi-autonomous driving really viable?

Tesla's Autopilot uses a combination of sensors and cameras to monitor the car's environment. The recent crash of Tesla Model S under Autopilot control has raised some serious concerns about the safety of autonomous driving features on Teslas, in particular, and all cars in general. The US National Highway Traffic Safety Administration (NHTSA)--the organization that offers the 5-star safety rating systems for new cars--is investigating the details of the unfortunate incident and may come up with more guidelines in this area, which many people believe is severely lacking in any real oversight. Much has already been written on the issue, but everything I've seen has ignored the key question that this incident has brought to our attention. Is it really reasonable or safe to offer a semi-autonomous driving mode, where a driver temporarily gives over complete control of an auto to computer-controlled systems within the car, but then needs to take it back under certain situations (such as a potential safety hazard)? To put it in the language of NHTSA and their guidelines for the development of autonomous driving technology, should there really be a Level 3 for autonomous driving?