Real-Time Indoor Object SLAM with LLM-Enhanced Priors

Abstract-- Object-level Simultaneous Localization and Mapping (SLAM), which incorporates semantic information for high-level scene understanding, faces challenges of under-constrained optimization due to sparse observations. Prior work has introduced additional constraints using commonsense knowledge, but obtaining such priors has traditionally been labor-intensive and lacks generalizability across diverse object categories. We address this limitation by leveraging large language models (LLMs) to provide commonsense knowledge of object geometric attributes, specifically size and orientation, as prior factors in a graph-based SLAM framework. These priors are particularly beneficial during the initial phase when object observations are limited. We implement a complete pipeline integrating these priors, achieving robust data association on sparse object-level features and enabling real-time object SLAM. Our system, evaluated on the TUM RGB-D and 3RScan datasets, improves mapping accuracy by 36.8% over the latest baseline. Object Simultaneous Localization and Mapping (SLAM) builds environment maps by identifying and localizing objects, and using this information to infer the robot's position. Unlike traditional feature-based SLAM, object-level representations are sparse, focusing on semantic object data. Comparing to semantic segmentation on dense representations, such sparsity improves computational efficiency and reduces storage requirements.