Autonomous Quadrotor Landing using Deep Reinforcement Learning

Abstract-- Landing an unmanned aerial vehicle (UAV) on a ground marker is an open problem despite the effort of the research community. Previous attempts mostly focused on the analysis of handcrafted geometric features and the use of external sensors in order to allow the vehicle to approach the land-pad. In this article, we propose a method based on deep reinforcement learning that only requires low-resolution images taken from a down-looking camera in order to identify the position of the marker and land the UAV on it. The proposed approach is based on a hierarchy of Deep Q-Networks (DQNs) used as high-level control policy for the navigation toward the marker. We implemented different technical solutions, such as the combination of vanilla and double DQNs, and a partitioned buffer replay. Using domain randomization we trained the vehicle on uniform textures and we tested it on a large variety of simulated and real-world environments. The overall performance is comparable with a state-of-the-art algorithm and human pilots. I. INTRODUCTION In the upcoming years an increasing number of autonomous systems will pervade urban and domestic environments. The next generation of Unmanned Aerial Vehicles (UAVs) requires high-level controllers in order to move in unstructured environments and perform multiple tasks. Recently a new application has been proposed, namely the use of quadrotors for the delivery of packages and goods. In this scenario the most delicate part is the identification of a ground marker and the vertical descent maneuver. Previous works used handcrafted features analysis and external sensors in order to identify the land-pad.