Spatial Traces: Enhancing VLA Models with Spatial-Temporal Understanding

-- Vision-Language-Action models have demonstrated remarkable capabilities in predicting agent movements within virtual environments and real-world scenarios based on visual observations and textual instructions. Although recent research has focused on enhancing spatial and temporal understanding independently, this paper presents a novel approach that integrates both aspects through visual prompting. We introduce a method that projects visual traces of key points from observations onto depth maps, enabling models to capture both spatial and temporal information simultaneously. The experiments in SimplerEnv show that the mean number of tasks successfully solved increased for 4% compared to SpatialVLA and 19% compared to TraceVLA. Furthermore, we show that this enhancement can be achieved with minimal training data, making it particularly valuable for real-world applications where data collection is challenging. The project page is available at https://ampiromax.github.io/ST-VLA . I. INTRODUCTION The rapid advancement of large language models (LLMs) has expanded their applications across various domains. One of the most promising areas of development is robotics and task planning [1], [2], [3], [4], [5], [6], where these models are being adapted to generate sequences of manipulator movements in a three-dimensional space based on environmental observations and task specifications [7], [8], [9], [10].