FLYINGTRUST: A Benchmark for Quadrotor Navigation Across Scenarios and Vehicles

Abstract--Visual navigation algorithms for quadrotors often exhibit a large variation in performance when transferred across different vehicle platforms and scene geometries, which increases the cost and risk of field deployment. T o support systematic early-stage evaluation, we introduce FL YINGTRUST, a high-fidelity, configurable benchmarking framework that measures how platform kinodynamics and scenario structure jointly affect navigation robustness. The benchmark pairs a diverse scenario library with a heterogeneous set of real and virtual platforms and prescribes a standardized evaluation protocol together with a composite scoring method that balances scenario importance, platform importance and performance stability. We use FL YINGTRUST to compare representative optimization-based and learning-based navigation approaches under identical conditions, performing repeated trials per platform-scenario combination and reporting uncertainty-aware metrics. The results reveal systematic patterns: navigation success depends predictably on platform capability and scene geometry, and different algorithms exhibit distinct preferences and failure modes across the evaluated conditions. These observations highlight the practical necessity of incorporating both platform capability and scenario structure into algorithm design, evaluation, and selection, and they motivate future work on methods that remain robust across diverse platforms and scenarios. NMANNED Aerial V ehicles (UA Vs) are aircraft operated without onboard human pilots, either by remote control or by preprogrammed flight plans [1]. By independently modulating the speeds of four motor-propeller units, a quadrotor can generate collective thrust for vertical motion and differential thrust and reaction torques for attitude control. These capabilities enable six degrees of freedom motion combined with fine low-speed control, which drive extensive adoption of quadrotors in precision agriculture, infrastructure inspection, high-resolution mapping, environmental monitoring and disaster response [2]-[11]. The benchmark of FL YINGTRUST is available at https://github.com/ The blue line represents the straight-line reference path, and the red curve is an example of a collision-free trajectory executed by a planner. Over the last decade, many high-performance visual navigation methods have been developed, ranging from classical optimization-based planners to recent learning-based approaches [12]-[15].