Gelina: Unified Speech and Gesture Synthesis via Interleaved Token Prediction

Early approaches used au-toregressive sequence modeling to map speech or text to motion sequences [19, 9], while diffusion-based generators now dominate for their ability to produce detailed, temporally consistent, and natural gestures [12, 10]. Other works explore discrete motion representations, enabling more controllable synthesis [8]. These models accept either speech or text as input and typically rely on speaker embeddings for multi-speaker modeling, which limits their generalization ability to speakers unseen during training. In contrast, Gelina generates both speech and gestures directly from text, and can also clone voice and gestural style through sequence continuation using a speech-gesture prompt, without relying on speaker embeddings. T ext-to-speech approaches: Lately, TTS has shifted toward data-driven methods, with notable advances in discrete code modeling [4, 5, 6].