Disney Research's AI system knows what a car sounds like

A picture may be worth a thousand words, but sound is just as important to how we experience the world as how we see it -- that's why a team at Disney Research is working on a computer vision system that can not only recognize what an image is, but how it sounds, too. In an initial study presented at the European Conference on Computer Vision, the group's system successfully managed to pair appropriate audio with images of doors closing, glasses clinking and vehicles driving down the road. Audio association might be easy for humans, but teaching a computer to do it is actually pretty challenging. Disney researchers trained AI to recognize the sound of images by feeding it a collection of videos demonstrating an object making a specific sound, but background noise, narration or sound made from other objects could easily confuse the system. If the system was fed samples with most of the uncorrelated sounds filtered out, however, it did a pretty good job of suggesting the right sound for each image.