MultiSCOPE: Disambiguating In-Hand Object Poses with Proprioception and Tactile Feedback

Abstract--In this paper, we propose a method for estimating in-hand object poses using proprioception and tactile feedback from a bimanual robotic system. Our method addresses the problem of reducing pose uncertainty through a sequence of frictional contact interactions between the grasped objects. As part of our method, we propose 1) a tool segmentation routine that facilitates contact location and object pose estimation, 2) a loss that allows reasoning over solution consistency between interactions, and 3) a loss to promote converging to object poses and contact locations that explain the external forcetorque experienced by each arm. We demonstrate the efficacy of our method in a task-based demonstration both in simulation and on a real-world bimanual platform and show significant improvement in object pose estimation over single interactions. These failure modes result in unreliable interactions between the robot and its environment when performing tasks such as Our method works by iteratively bringing the two objects into tool use and assembly. These filters visual feedback to identify objects in the scene and estimate exploit mutual information between known object geometries their poses [6, 9, 11, 12, 15, 19, 21, 25, 26, 29].