Unsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures using Spatial Information

UNSUPERVISED DEEP CLUSTERING FOR SOURCE SEPARATION: DIRECT LEARNING FROM MIXTURES USING SPATIAL INFORMATION Efthymios Tzinis ] Shrikant Venkataramani ] Paris Smaragdis ][ ] University of Illinois at Urbana-Champaign, Department of Computer Science [ Adobe Research ABSTRACT We present a monophonic source separation system that is trained by only observing mixtures with no ground truth separation information. We use a deep clustering approach which trains on multi-channel mixtures and learns to project spectrogram bins to source clusters that correlate with various spatial features. We show that using such a training process we can obtain separation performance that is as good as making use of ground truth separation information. Once trained, this system is capable of performing sound separation on monophonic inputs, despite having learned how to do so using multi-channel recordings. Index Terms -- Deep clustering, source separation, unsupervised learning 1. INTRODUCTION A central problem when designing source separation systems is that of defining what constitutes a source.