Semi-blind source separation with multichannel variational autoencoder

This paper proposes a multichannel source separation method called the multichannel variational autoencoder (MVAE), which uses a conditional VAE (CVAE) to model and estimate the power spectrograms of the sources in a mixture. By training the CVAE using the spectrograms of training examples with source-class labels, we can use the trained decoder distribution as a universal generative model that is able to generate spectrograms conditioned on a specified class label. By treating the latent space variables and the class label as the unknown parameters of this generative model, we can develop a convergence-guaranteed semi-blind source separation algorithm that consists of iteratively estimating the power spectrograms of the underlying sources as well as the separation matrices. Through experimental evaluations, our MVAE showed higher separation performance than a baseline method.