Zero-Shot 3D Visual Grounding from Vision-Language Models
Li, Rong, Li, Shijie, Kong, Lingdong, Yang, Xulei, Liang, Junwei
–arXiv.org Artificial Intelligence
Existing approaches typically rely on labeled 3D data and predefined categories, limiting scalability to open-world settings. W e present SeeGround, a zero-shot 3DVG framework that leverages 2D Vision-Language Models (VLMs) to bypass the need for 3D-specific training. T o bridge the modality gap, we introduce a hybrid input format that pairs query-aligned rendered views with spatially enriched textual descriptions. Our framework incorporates two core components: a Perspective Adaptation Module that dynamically selects optimal viewpoints based on the query, and a Fusion Alignment Module that integrates visual and spatial signals to enhance localization precision. Extensive evaluations on ScanRefer and Nr3D confirm that SeeGround achieves substantial improvements over existing zero-shot baselines - outperforming them by 7.7% and 7. 1%, respectively - and even rivals fully supervised alternatives, demonstrating strong generalization under challenging conditions.
arXiv.org Artificial Intelligence
May-29-2025