Struct2D: APerception-Guided Framework for Spatial Reasoning in MLLMs

Neural Information Processing Systems 

Unlocking spatial reasoning in Multimodal Large Language Models (MLLMs) is crucial for enabling intelligent interaction with 3D environments. While prior ef ask: forts can often MLLMs rely on reason explicit about 3D 3D inputs space or specialized using only model structur architectures, ed 2D represenwe tations derived from perception? We introduce Struct2D, a perception-guided prompting marks and object-centric framework that metadata, combines optionally bird's-eye-vie incorporating w (BEV) egocentric images with keyframes object when needed. Using Struct2D, we conduct an in-depth zero-shot analysis of closed-source spatial reasoning MLLMs abilities (e.g.