Towards Robust Direct Perception Networks for Automated Driving