Goto

Collaborating Authors

 heme


Pheme: Efficient and Conversational Speech Generation

arXiv.org Artificial Intelligence

In recent years, speech generation has seen remarkable progress, now achieving one-shot generation capability that is often virtually indistinguishable from real human voice. Integrating such advancements in speech generation with large language models might revolutionize a wide range of applications. However, certain applications, such as assistive conversational systems, require natural and conversational speech generation tools that also operate efficiently in real time. Current state-of-the-art models like VALL-E and SoundStorm, powered by hierarchical neural audio codecs, require large neural components and extensive training data to work well. In contrast, MQTTS aims to build more compact conversational TTS models while capitalizing on smaller-scale real-life conversational speech data. However, its autoregressive nature yields high inference latency and thus limits its real-time usage. In order to mitigate the current limitations of the state-of-the-art TTS models while capitalizing on their strengths, in this work we introduce the Pheme model series that 1) offers compact yet high-performing models, 2) allows for parallel speech generation of 3) natural conversational speech, and 4) it can be trained efficiently on smaller-scale conversational data, cutting data demands by more than 10x but still matching the quality of the autoregressive TTS models. We also show that through simple teacher-student distillation we can meet significant improvements in voice quality for single-speaker setups on top of pretrained Pheme checkpoints, relying solely on synthetic speech generated by much larger teacher models. Audio samples and pretrained models are available online.


Riveter: Measuring Power and Social Dynamics Between Entities

arXiv.org Artificial Intelligence

Riveter provides a complete easy-to-use pipeline for analyzing verb connotations associated with entities in text corpora. We prepopulate the package with connotation frames of sentiment, power, and agency, which have demonstrated usefulness for capturing social phenomena, such as gender bias, in a broad range of corpora. For decades, lexical frameworks have been foundational tools in computational social science, digital humanities, and natural language processing, facilitating multifaceted analysis of text corpora. But working with verb-centric lexica specifically requires natural language processing skills, reducing their accessibility to other researchers. By organizing the language processing pipeline, providing complete lexicon scores and visualizations for all entities in a corpus, and providing functionality for users to target specific research questions, Riveter greatly improves the accessibility of verb lexica and can facilitate a broad range of future research.


Magic Pills, Machine-Learning Skincare, and the Future of Health

#artificialintelligence

What it is: Self-described by its MIT creators as "the world's first cellular health product informed by genomics," Basis by Elysium Health is a mail-order daily supplement that's been making waves in the rapidly emerging field of life-extension science. The claim is not immortality but simply the possibility of extending one's vital years, by putting off of cancer, heart disease, diabetes, Alzheimer's, and other afflictions of age. Hard to say: Though the supplement is some 25 years in the making, it's been human-tested for less than five, including on Elysium Health cofounder Leonard Guarente, who also serves as the director of MIT's center for aging research. That said, the company has an impressive roster of Nobel Prize winners on its scientific advisory board and has attracted more than $25 million in funding. What's the sell: Two pills a day purportedly target DNA repair, cellular detoxification, energy production, and protein function by converting nicotinamide riboside into nicotinamide adenine dinucleotide (NAD), a coenzyme critical to metabolism that diminishes with age.