Amazon's voice-synthesizing AI mimics shifts in tempo, pitch, and volume

Voice assistants like Alexa convert written words into speech using text-to-speech systems, the most capable of which tap AI to verbalize from scratch rather than stringing together prerecorded snippets of sounds. Neural text-to-speech systems, or NTTS, tend to produce more natural-sounding speech than conventional models, but arguably their real value lies in their adaptability, as they're able to mimic the prosody of a recording, or its shifts in tempo, pitch, and volume. In a paper ("Fine-Grained Robust Prosody Transfer for Single-Speaker Neural Text-to-Speech") presented at this year's Interspeech conference in Graz, Austria, Amazon scientists investigated prosody transfer with a system that enabled them to choose voices in recordings while preserving the original inflections. They say it significantly improved on past attempts, which generally haven't adapted well to input voices they haven't encountered before. To this end, the team's system leveraged prosodic features that are easier to normalize than the raw spectrograms (representations of changes in signal frequency over time) typically ingested by neural text-to-speech networks.