Artificial Intelligence Can Create Sound Tracks for Silent Videos

Researchers Ghose and Prevost created a deep learning algorithm which, given a silent video, can generate a realistic sounding synchronized soundtrack. Frequently, movies had added sound effects which were not recorded during after the recording to make it feel more realistic in a process called "Foley". Researchers at the university of Texas turned to deep learning to automate this process. They trained a neural network on 12 popular movie events where directors frequently add Foley effects. Their neural network classifies the class of the sound to generate, and also has a sequential network that generates the sound.