Relating Sound and Sight in Simulated Environments

What is it about?

When you hear a sound, it can be hard to distinguish if what you hear is true to the sound source and its reflections from the environment (i.e., reverberation), or whether it is a produced sound mimicking this. Vision can help resolve some of this ambiguity when discerning sound source and reflections, as it can reveal information about the environment's size or the objects in it, which helps infer the reverberation of the sound. This study investigated whether the visual environment helps us understand where a sound was produced by inferring its reverberation. Participants were presented with sets of visual panoramas and reverberant sounds, and were tasked to judge whether they originated from the same (congruent) or different (incongruent) locations. Overall, participants responded with audiovisual matches 2.6 times more to congruent pairs than to incongruent pairs. This suggests that visual information is useful in forming expectations of reverberation, and thus can help in understanding the sound source and reverberation separately.

Featured Image