Waveform generation for text-to-speech synthesis using pitch-synchronous multi-scale generative adversarial networks

ABSTRACT The state-of-the-art in text-to-speech synthesis has recently improved considerably due to novel neural waveform generation methods, such as WaveNet. However, these methods suffer from their slow sequential inference process, while their parallel versions are difficult to train and even more expensive computationally. Meanwhile, generative adversarial networks (GANs) have achieved impressive results in image generation and are making their way into audio applications; parallel inference is among their lucrative properties. Listening test results show that while direct waveform generation with GAN is still far behind WaveNet, a GAN-based glottal excitation model can achieve quality and voice similarity on par with a WaveNet vocoder. Index Terms-- Neural vocoding, text-to-speech, GAN, glottal excitation model 1. INTRODUCTION Recent advances in deep learning have led to text-to-speech (TTS) systems achieving near-human naturalness [1]. This is partially due to neural sequence-to-sequence mapping methods that can learn to align and map between input text and output acoustic feature sequences [2].