Workflow
Adaptation to Intrinsic Dependence in Diffusion Language Models
Diffusion language models (DLMs) have recently emerged as a promising alternative to autoregressive (AR) approaches, enabling parallel token generation beyond a rigid left-to-right order. Despite growing empirical success, the theoretical understanding of how unmasking schedules -- which specify the order and size of unmasked tokens during sampling -- affect generation quality remains limited. In this work, we introduce a distribution-agnostic unmasking schedule for DLMs that adapts to the (unknown) dependence structure of the target data distribution, without requiring any prior knowledge or hyperparameter tuning. In contrast to prior deterministic procedures that fix unmasking sizes, our method randomizes the number of tokens revealed at each iteration. We show that, for two specific parameter choices, the sampling convergence guarantees -- measured by Kullback-Leibler (KL) divergence -- scale as $\widetilde O(\mathsf{TC}/K)$ and $\widetilde O(\mathsf{DTC}/K)$ respectively. Here, $K$ is the number of iterations, and $\mathsf{TC}$ and $\mathsf{DTC}$ are the total correlation and dual total correlation of the target distribution, capturing the intrinsic dependence structure underlying the data. Importantly, our guarantees hold in the practically relevant parallel-sampling regime $K
- Workflow (0.93)
- Research Report > New Finding (0.87)
- North America > United States > Virginia (0.04)
- Asia > China (0.04)
- Workflow (0.68)
- Research Report > Experimental Study (0.46)
- Information Technology > Security & Privacy (1.00)
- Health & Medicine > Therapeutic Area > Oncology (0.46)
- North America > United States > Minnesota (0.04)
- Europe > United Kingdom > England > Oxfordshire > Oxford (0.04)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- Asia > China > Hong Kong (0.04)
- Health & Medicine (0.46)
- Energy (0.46)
- Government (0.46)
- North America > United States > California > San Diego County > San Diego (0.04)
- North America > United States > California > Los Angeles County > Long Beach (0.04)
- Asia > China > Hong Kong (0.04)
- Education (0.67)
- Government > Regional Government > North America Government > United States Government (0.46)
- North America > United States > Missouri > Boone County > Columbia (0.13)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.13)
- Europe > Switzerland > Basel-City > Basel (0.04)
- (2 more...)
- Research Report (1.00)
- Workflow (0.67)
- Europe > United Kingdom > England (0.04)
- Asia > Middle East > Israel (0.04)
- Asia > Japan (0.04)
- (4 more...)
- Government (1.00)
- Health & Medicine (0.68)
- Banking & Finance (0.67)
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks (0.46)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (0.46)
- Asia > China > Beijing > Beijing (0.04)
- North America > United States (0.04)
- North America > Canada (0.04)
- Asia > Middle East > Jordan (0.04)
- Research Report (0.46)
- Workflow (0.46)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (0.92)
- Workflow (0.67)
- Research Report > Experimental Study (0.93)
- Workflow (0.67)
- North America > United States > California > Los Angeles County > Los Angeles (0.04)
- North America > United States > Illinois (0.04)
- North America > United States > District of Columbia > Washington (0.04)
- (3 more...)
- Research Report > Experimental Study (0.93)
- Workflow (0.68)
- Overview (0.67)
- Health & Medicine > Therapeutic Area (1.00)
- Information Technology (0.93)
- Leisure & Entertainment > Sports (0.92)
- Government (0.67)