Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations

Sounds are generally much more distinctive compared Research in the field of neuroscience shows that with multiple with visual events. For some specific tasks related to physical cues from a diverse range of sensory modalities comes properties estimation, the sound alone is reliable to guide enhanced behavioral performance towards faster response, a robot and measure its performance [12]. For others, it more accurate movement, and a better sense of stimulus [1]. may be informative but not sufficient, e.g., a classification When presented with multiple modalities, e.g., a combination of objects that share common auditory properties [13], or of auditory, haptic, and visual perception, an observer will precise control of a water-pouring robot [14]. In this case, make the assumption of unity that decides whether the sounds are supposed to fuse with other sensory inputs to multimodal information originates from a common source or present a much more robust description of states, or to from some separated objects and events [2]. The perception scaffold the agent's exploration. of unity arises when the perceiver assumes that a physical There are more chances that sound is abundantly distributed event is redundantly expressed and sensed across diverse while hardly considered for general manipulations modalities, and decisions are commonly made based on the due to the facts that 1) vision is content-rich and is thus temporal and spatial consistency of information [3], or on sufficient for traditional planning-based robots so the sound semantic congruence factors [1].