Artificial Sound Effects Have Now Entered the Uncanny Valley

Using machine learning, researchers from MIT have developed a system that produces sound effects that are so realistic they even fool human listeners. The new algorithm, developed by researchers from MIT's Computer Science and Artificial Intelligence Laboratory, can predict the precise acoustical qualities of a sound, and then simulate it in an extremely realistic way. When analyzing a silent video clip, such as an object being hit by a drumstick, the system can produce a sound for the hit that's realistic enough to fool human listeners. To make it work, PhD student Andrew Owens and his team applied a technique known as "deep learning" that enables computers to pick out important patterns buried in massive amounts of raw data completely autonomously. Over the course of several months, the researchers recorded about 1,000 videos of an estimated 46,000 sounds that represented an array of objects being hit, scraped, and prodded by a drumstick.