Goto

Collaborating Authors

 virt&r


UAV See, UGV Do: Aerial Imagery and Virtual Teach Enabling Zero-Shot Ground Vehicle Repeat

Fisker, Desiree, Krawciw, Alexander, Lilge, Sven, Greeff, Melissa, Barfoot, Timothy D.

arXiv.org Artificial Intelligence

-- This paper presents Virtual T each and Repeat (VirT&R): an extension of the T each and Repeat (T&R) framework that enables GPS-denied, zero-shot autonomous ground vehicle navigation in untraversed environments. VirT&R leverages aerial imagery captured for a target environment to train a Neural Radiance Field (NeRF) model so that dense point clouds and photo-textured meshes can be extracted. The NeRF mesh is used to create a high-fidelity simulation of the environment for piloting an unmanned ground vehicle (UGV) to virtually define a desired path. The mission can then be executed in the actual target environment by using NeRF-generated point cloud submaps associated along the path and an existing LiDAR T each and Repeat (L T&R) framework. We benchmark the repeatability of VirT&R on over 12 km of autonomous driving data using physical markings that allow a sim-to-real lateral path-tracking error to be obtained and compared with L T&R. VirT&R achieved measured root mean squared errors (RMSE) of 19.5 cm and 18.4 cm in two different environments, which are slightly less than one tire width (24 cm) on the robot used for testing, and respective maximum errors were 39.4 cm and 47.6 cm. This was done using only the NeRF-derived teach map, demonstrating that VirT&R has similar closed-loop path-tracking performance to L T&R but does not require a human to manually teach the path to the UGV in the actual environment. I. INTRODUCTION Enabling a higher level of autonomous navigation in remote, harsh, and potentially hazardous environments is a critical objective for many Unmanned Ground V ehicle (UGV) operations, as minimizing human presence in such scenarios reduces risk and lowers costs. Visual Teach and Repeat (VT&R) [1], is a complete autonomy stack that enables long-range navigation along previously taught routes, demonstrated on a UGV with 3D-LiDAR [2]-[4], Radar [5], and RGB vision sensors [1], as well as on a UA V with an RGB vision sensor [6], [7]. While Teach and Repeat (T&R) has demonstrated considerable success, it currently requires a human operator to manually guide the vehicle in the environment during the teaching phase to create a map and ensure traversability.