SonicSense: Object Perception from In-Hand Acoustic Vibration

By shaking a container, we can tell its inventory status from the generated acoustic vibrations, such as the quantity and geometry of the objects inside. Similarly, we can identify the material and geometry of the entire object through multiple tappings. Human hands are equipped with high-frequency skin vibrations to help capture such complex object properties [1]. However, despite the significance of acoustic vibrations for tactile perception, equipping robot manipulators with acoustic vibration sensing capability for rich object perception remains difficult [2, 3, 4, 5, 6]. Though previous research has explored placing air microphones near robot platforms to estimate liquid height [7] and pouring amounts [8], classify object materials [9] and categories [10, 11, 12], air microphones mainly capture Figure 1: SonicSense enables container sound waves transmitted through air, leading to noisy inventory status differentiation, heterogeneous signals with ambient noises. On the other hand, contact material prediction, 3D microphones only sense the acoustic vibrations caused by shape reconstruction, and object reidentification on a diverse set of 83 realworld physical contact. Past work has studied contact microphones objects.