gelina
Gelina: Unified Speech and Gesture Synthesis via Interleaved Token Prediction
Guichoux, Téo, Lemerle, Théodor, Mehta, Shivam, Beskow, Jonas, Henter, Gustav Eje, Soulier, Laure, Pelachaud, Catherine, Obin, Nicolas
Early approaches used au-toregressive sequence modeling to map speech or text to motion sequences [19, 9], while diffusion-based generators now dominate for their ability to produce detailed, temporally consistent, and natural gestures [12, 10]. Other works explore discrete motion representations, enabling more controllable synthesis [8]. These models accept either speech or text as input and typically rely on speaker embeddings for multi-speaker modeling, which limits their generalization ability to speakers unseen during training. In contrast, Gelina generates both speech and gestures directly from text, and can also clone voice and gestural style through sequence continuation using a speech-gesture prompt, without relying on speaker embeddings. T ext-to-speech approaches: Lately, TTS has shifted toward data-driven methods, with notable advances in discrete code modeling [4, 5, 6].
- Europe > France > Île-de-France > Paris > Paris (0.04)
- North America > United States > Illinois > Cook County > Chicago (0.04)
- Europe > Switzerland (0.04)
- (3 more...)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (0.68)
A robot reporter chasing down stories about alien cats: how Times & Galaxy nails journalism
A game about a robot becoming a journalist feels a bit on the nose right now, in the midst of stories about writers being replaced by AI. But Ben Gelinas, director of Times & Galaxy, says it was never his intention to make a point about the rise of artificial intelligence. The intention was to focus on journalism itself. "You can't write for everybody, and I'm trying to show that." You play as Reporterbot, the first ever robot reporter for the Times & Galaxy, a space "holopaper" that's produced aboard a starship.
- Media > News (1.00)
- Leisure & Entertainment > Games > Computer Games (0.31)