LiftFeat: 3D Geometry-Aware Local Feature Matching

-- Robust and efficient local feature matching plays a crucial role in applications such as SLAM and visual localization for robotics. Despite great progress, it is still very challenging to extract robust and discriminative visual features in scenarios with drastic lighting changes, low texture areas, or repetitive patterns. In this paper, we propose a new lightweight network called LiftF eat, which lifts the robustness of raw descriptor by aggregating 3D geometric feature. Specifically, we first adopt a pre-trained monocular depth estimation model to generate pseudo surface normal label, supervising the extraction of 3D geometric feature in terms of predicted surface normal. Integrating such 3D geometric feature enhances the discriminative ability of 2D feature description in extreme conditions. Extensive experimental results on relative pose estimation, homography estimation, and visual localization tasks, demonstrate that our LiftFeat outperforms some lightweight state-of-the-art methods. I. INTRODUCTION Local feature matching between images is critical for many core robotic tasks, including Structure from Motion (SfM) [1], [2], [3], Simultaneous Localization and Mapping (SLAM) [4], [5], [6], [7], and visual localization [8], [9], [10], [11]. In practical applications, there are some scenes with extreme conditions, such as significant variation of illumination, and the presence of textureless or repetitive patterns.