High-quality nonparallel voice conversion based on cycle-consistent adversarial network

ABSTRACT Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data. In this paper, we propose using a cycle-consistent adversarial network (CycleGAN) for nonparallel data-based VC training. A CycleGAN is a generative adversarial network (GAN) originally developed for unpaired image-to-image translation. A subjective evaluation of inter-gender conversion demonstrated that the proposed method significantly outperformed a method based on the Merlin open source neural network speech synthesis system (a parallel VC system adapted for our setup) and a GAN-based parallel VC system. This is the first research to show that the performance of a nonparallel VC method can exceed that of state-of-the-art parallel VC methods. Index Terms-- Voice conversion, deep learning, cycle-consistent adversarial network, generative adversarial network 1. INTRODUCTION Voice conversion (VC) is a technique for modifying the speech signals of a source speaker to match those of a target speaker so that it sounds as if the target speaker had spoken while keeping the linguistic information unchanged [1, 2].