Speech Synthesis

Machine learning has played a very important role in the development of technology that has a large impact on our everyday lives. However, machine learning is also influencing the direction of technology that is not as commonplace. Text to speech technology is a prime example. Text to speech technology predates machine learning by over a century. However, machine learning has made the technology more reliable than ever.

Training powerful text to speech models requires sufficiently powerful hardware. A recent study published by OpenAI drives the point home -- it found that since 2012, the amount of compute used in the largest runs grew by more than 300,000 times. In pursuit of less demanding models, researchers at IBM developed a new lightweight and modular method for speech synthesis. They say it's able to synthesize high-quality speech in real time by learning different aspects of a speaker's voice, making it possible to adapt to new speaking styles and voices with small amounts of data. "Recent advances in deep learning are dramatically improving the development of Text-to-Speech (TTS) systems through more effective and efficient learning of voice and speaking styles of speakers and more natural generation of high-quality output speech," wrote IBM researchers Zvi Kons, Slava Shechtman, and Alex Sorin in a blog post accompanying a preprint paper presented at Interspeech 2019.

The authors of this paper are from Google. They present a neural network for generating raw audio waves. Their model is fully probabilistic and autoregressive, and it generates state-of-the-art text-to-speech results for both English and Mandarin. WaveNet is an audio generative model based on the PixelCNN. In this generative model, each audio sample is conditioned on the previous audio sample.

Machine learning has played a very important role in the development of technology that has a large impact on our everyday lives. However, machine learning is also influencing the direction of technology that is not as commonplace. Text to speech technology is a prime example. Text to speech technology predates machine learning by over a century. However, machine learning has made the technology more reliable than ever.

ICYMI: Earlier this summer we broke new ground with RealTalk, a speech synthesis system created by Machine Learning Engineers at Dessa. With their AI-powered text-to-speech system, the team managed to replicate the voice of Joe Rogan, a podcasting legend known for his irreverent takes on consciousness, sports and technology. On top of that, their recreation of Rogan's voice is the most realistic AI voice that's been released to date. If you haven't heard the voice yet, you should. Here's the video we shared on YouTube featuring a medley of their faux Rogan's musings: Since then, the public's response to the work has wowed us.

A text-to-speech system, which converts written text into synthesized speech, is what allows Alexa to respond verbally to requests or commands. Through a service called Amazon Polly, text-to-speech is also a technology that Amazon Web Services offers to its customers. Last year, both Alexa and Polly evolved toward neural-network-based text-to-speech systems, which synthesize speech from scratch, rather than the earlier unit-selection method, which strung together tiny snippets of pre-recorded sounds. In user studies, people tend to find speech produced by neural text-to-speech (NTTS) systems more natural-sounding than speech produced by unit selection. But the real advantage of NTTS is its adaptability, something we demonstrated last year in our work on changing the speaking style ("newscaster" versus "neutral") of an NTTS system.

Google LLC today updated its Cloud Text-to-Speech service with new languages and voices in order to make it useful to more of its customers. Google Cloud Text-to-Speech is intended to help companies develop better conversational interfaces for the services they supply. It works by transforming written text into artificial speech that's spoken in realistic human voices. With the service, Google is targeting three main markets: voice response systems for call centers; "internet of things" products such as car infotainment systems, TVs and robots; and applications such as podcasts and audiobooks, which convert text into speech. In a blog post, Google product manager Dan Aharon said Cloud Text-to-Speech is getting 12 new languages or variants, including Czech, English (India), Filipino, Finnish, Greek, Hindi, Hungarian, Indonesian, Mandarin Chinese (China), Modern Standard Arabic and Vietnamese.

Cloning a voice typically requires collecting hours of recorded speech to build a dataset then using the dataset to train a new voice model. A new Github project introduces a remarkable Real-Time Voice Cloning Toolbox that enables anyone to clone a voice from as little as five seconds of sample audio. This Github repository was open sourced this June as an implementation of the paper Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (SV2TTS) with a vocoder that works in real-time. The project was developed by Corentin Jemine, who got his Masters in Data Science at the University of Liège and works as a machine learning engineer at Resemble AI in Toronto. Users input a short voice sample and the model -- trained only during playback time -- can immediately deliver text-to-speech utterances in the style of the sampled voice.

The authors of this paper are from Google. They present a neural network for generating raw audio waves. Their model is fully probabilistic and autoregressive, and it generates state-of-the-art text-to-speech results for both English and Mandarin. WaveNet is an audio generative model based on the PixelCNN. In this generative model, each audio sample is conditioned on the previous audio sample.