A Reparameterized Discrete Diffusion Model for Text Generation

We derive an alternative yet equivalent However, there are noticeably fewer success cases in employing formulation of the sampling from discrete diffusion models for large-scale text generation diffusion processes and leverage this insight to tasks. This is possibly due to the discrete nature of natural develop a family of reparameterized discrete diffusion languages, while most conventional diffusion models focus models. The derived generic framework is on continuous-valued contents. To bridge the discrepancy, highly flexible, offers a fresh perspective of the recent work aims at conducting the diffusion process over token generation process in discrete diffusion models, embeddings so that the continuous diffusion models can and features more effective training and decoding be applied to discrete texts (Li et al., 2022; Gong et al., 2022; techniques. We conduct extensive experiments Strudel et al., 2022; Dieleman et al., 2022) or logits (Han to evaluate the text generation capability of our et al., 2022; Richemond et al., 2022). Nevertheless, these model, demonstrating significant improvements approaches often require designing a well-crafted rounding over existing diffusion models.