Why Speech Separation is Such a Difficult Problem to Solve

You are talking on the phone, or recording an audio, or just speaking to voice assistants like Google Assistant, Cortana, or Alexa. But the person on the other side of the call cannot hear you because you are in a crowded place, the recorded audio has a lot of background noise, or the "Hey, Alexa" call wasn't picked up by your device because someone else started speaking. All of these problems related to separating voices, informally referred to as the "cocktail party problem", have been addressed using artificial intelligence and deep learning methods in recent years. But still, separating and inferring multiple simultaneous voices is a difficult problem to completely solve. To start, speech separation is extracting speech of the "wanted speaker" or "speaker of interest" from the overlapping mixture of speech from other speakers, also referred to as'noise'.