End-to-end models for raw audio generation are a challenge, specially if they have to work with non-parallel data, which is a desirable setup in many situations. Voice conversion, in which a model has to impersonate a speaker in a recording, is one of those situations. In this paper, we propose Blow, a single-scale normalizing flow using hypernetwork conditioning to perform many-to-many voice conversion between raw audio. Blow is trained end-to-end, with non-parallel data, on a frame-by-frame basis using a single speaker identifier. We show that Blow compares favorably to existing flow-based architectures and other competitive baselines, obtaining equal or better performance in both objective and subjective evaluations. We further assess the impact of its main components with an ablation study, and quantify a number of properties such as the necessary amount of training data or the preference for source or target speakers.

We are very grateful to all reviewers for their positive assessment of our work and their constructive feedback. We have acted in two ways to improve our work. Spoofing metric, which is 99.3% for the considered data set and split. Finally, one reviewer points out that we should compare to "AUTOVC: Zero-shot V oice Style Transfer with Only

Using the invertibility(forward-backward) of flow-based model to do voice conversion is an overall clever idea. The novelty in machine learning/deep learning is limited. The session 4 is more like an architecture tuning summary of Glow/WaveGlow. Also the quality of posted audio samples and subjective evaluation(both naturalness and similarity) need to be improved. So there is no information bottleneck like auto-encoder based models(e.g.

End-to-end models for raw audio generation are a challenge, specially if they have to work with non-parallel data, which is a desirable setup in many situations. Voice conversion, in which a model has to impersonate a speaker in a recording, is one of those situations. In this paper, we propose Blow, a single-scale normalizing flow using hypernetwork conditioning to perform many-to-many voice conversion between raw audio. Blow is trained end-to-end, with non-parallel data, on a frame-by-frame basis using a single speaker identifier. We show that Blow compares favorably to existing flow-based architectures and other competitive baselines, obtaining equal or better performance in both objective and subjective evaluations.