GPA-RAM: Grasp-Pretraining Augmented Robotic Attention Mamba for Spatial Task Learning

Abstract--T ask failures in prior fine-grained robotic manipulation methods often stem from suboptimal initial grasping, which is critical for subsequent manipulation and reducing the requirement for complex pose adjustments. T o address this, we propose Grasp-Pretraining Augmentation (GPA)--a general multi-modal learning framework that enhances grasp perception without additional grasp pose data collection and labeling. GPA achieves evident enhancement on RLBench multi-task benchmark (from 79.3% to 84.2%) and ALOHA bimanual manipulation tasks (from 86%/16% to 98%/38%). Although GPA enhances fine-grained grasping performance by leveraging increased model capacity, it incurs computational latency and hinders real-time deployment. T o mitigate this limitation, we propose Robotic Attention Mamba (RAM). Our unified GPA-RAM framework balances model capacity with efficiency and applies to both discrete and continuous action generation. GPA-RAM demonstrates superior performance across four robotic systems with diverse camera configurations in both simulation and the real world. Compared with previous state-of-the-art methods, it improves average success rates by 8.2% over RVT2 (from 79.3% to 87.5%) and 2.6% over ARP This work provides a framework for developing robotic systems that are simultaneously precise and responsive. The project and code are at https://gpa-ram.github.io/ These authors contributed equally to this work. This work was supported by National Natural Science Foundation of China (No. 62473007), and Shenzhen Innovation in Science and Technology Foundation for The Excellent Y outh Scholars (No. RCYX20231211090248064). (Corresponding author: Mengyuan Liu.) Juyi Sheng, Peiming Li and Mengyuan Liu are with the State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School, Shenzhen, 518055, China (email: logss2024@stu.pku.edu.cn;