No Need to Look! Locating and Grasping Objects by a Robot Arm Covered with Sensitive Skin

This work has been submitted to the IEEE for possible publication. No Need to Look! Locating and Grasping Objects by a Robot Arm Covered with Sensitive Skin Abstract-- Locating and grasping of objects by robots is typically performed using visual sensors. Haptic feedback from contacts with the environment is only secondary if present at all. The main novelty lies in the use of contacts over the complete surface of a robot manipulator covered with sensitive skin. The search is divided into two phases: (1) coarse workspace exploration with the complete robot surface, followed by (2) precise localization using the end-effector equipped with a force/torque sensor . We systematically evaluated this method in simulation and on the real robot, demonstrating that diverse objects can be located, grasped, and put in a basket. The overall success rate on the real robot for one object was 85.7% with failures mainly while grasping specific objects. The method using whole-body contacts is six times faster compared to a baseline that uses haptic feedback only on the end-effector . We also show locating and grasping multiple objects on the table. This method is not restricted to our specific setup and can be deployed on any platform with the ability of sensing contacts over the entire body surface. This work holds promise for diverse applications in areas with challenging visual perception (due to lighting, dust, smoke, occlusion) such as in agriculture when fruits or vegetables need to be located inside foliage and picked. Perception for robot manipulation has been dominated by visual inputs from cameras (RGB) or depth cameras (RGB-D). Classical methods have been used for object segmentation and pose and shape estimation to feed the synthesis of grasp proposals for a robot hand (e.g., [1]).