Deep listening: The neural network learning to hear you in a crowd

The human auditory system gives us the extraordinary ability to converse above the chatter of a lively cocktail party. Selective listening in such conditions is an extremely challenging task for computers, and has been the holy grail of speech processing for more than 50 years. Previously, no practical method existed in the case of single channel mixtures of speech, especially when the speakers are unknown, but now Mitsubishi Electric Research Labs (MERL) are addressing the problem of acoustic source separation with a deep learning framework they call "deep clustering". At the Deep Learning Summit in Boston last month John Hershey, Senior Principal Research Scientist at MERL, presented'Cracking the Cocktail Party Problem: Deep Clustering for Speech Separation' and shared their breakthrough, using their deep clustering network to assign embedding vectors to different sonic elements of the noisy signal. With this technology, MERL are on the verge of solving the general audio separation problem, opening up a new era in spontaneous human-machine communication.