End to End Vehicle Lateral Control Using a Single Fisheye Camera

Abstract-- Convolutional neural networks are commonly used to control the steering angle for autonomous cars. Most of the time, multiple long range cameras are used to generate lateral failure cases. In this paper we present a novel model to generate this data and label augmentation using only one short range fisheye camera. We present our simulator and how it can be used as a consistent metric for lateral end-to-end control evaluation. Experiments are conducted on a custom dataset corresponding to more than 10000 km and 200 hours of open road driving. Finally we evaluate this model on real world driving scenarios, open road and a custom test track with challenging obstacle avoidance and sharp turns. In our simulator based on real-world videos, the final model was capable of more than 99% autonomy on urban road. The ultimate goal for autonomous vehicles is to drive in any environment without any human input. To achieve this, autonomous cars have to analyze their environment using data coming from different sensors and control the car accordingly. In the most common approach, this task is cut into different modules then fed into a rule-based control algorithm which actually drives the car.