Disambiguate Gripper State in Grasp-Based Tasks: Pseudo-Tactile as Feedback Enables Pure Simulation Learning

-- Grasp-based manipulation tasks are fundamental to robots interacting with their environments, yet gripper state ambiguity significantly reduces the robustness of imitation learning policies for these tasks. Data-driven solutions face the challenge of high real-world data costs, while simulation data, despite its low costs, is limited by the sim-to-real gap. We identify the root cause of gripper state ambiguity as the lack of tactile feedback. T o address this, we propose a novel approach employing pseudo-tactile as feedback, inspired by the idea of using a force-controlled gripper as a tactile sensor . This method enhances policy robustness without additional data collection and hardware involvement, while providing a noise-free binary gripper state observation for the policy and thus facilitating pure simulation learning to unleash the power of simulation. Experimental results across three real-world grasp-based tasks demonstrate the necessity, effectiveness, and efficiency of our approach. Videos are available on Project Page. Grasp-based manipulation spans a wide range of tasks, from simple pick-and-place [1], [2] to more complex interactions like tool usage [3], [4], making it a fundamental capability for robots to engage with the environment. One promising way to teach robots these skills is imitation learning (IL) [5], [6], which enables robots to learn directly from expert demonstrations through supervised learning. The efficacy of IL is heavily dependent on the quantity and quality of the provided demonstrations.