Tong, Vinh
Symmetry-Preserving Diffusion Models via Target Symmetrization
Tong, Vinh, Ye, Yun, Hoang, Trung-Dung, Liu, Anji, Broeck, Guy Van den, Niepert, Mathias
Diffusion models are powerful tools for capturing complex distributions, but modeling data with inherent symmetries, such as molecular structures, remains challenging. Equivariant denoisers are commonly used to address this, but they introduce architectural complexity and optimization challenges, including noisy gradients and convergence issues. We propose a novel approach that enforces equivariance through a symmetrized loss function, which applies a time-dependent weighted averaging operation over group actions to the model's prediction target. This ensures equivariance without explicit architectural constraints and reduces gradient variance, leading to more stable and efficient optimization. Our method uses Monte Carlo sampling to estimate the average, incurring minimal computational overhead. We provide theoretical guarantees of equivariance for the minimizer of our loss function and demonstrate its effectiveness on synthetic datasets and the molecular conformation generation task using the GEOM-QM9 dataset. Experiments show improved sample quality compared to existing methods, highlighting the potential of our approach to enhance the scalability and practicality of equivariant diffusion models in generative tasks.
Learning to Discretize Denoising Diffusion ODEs
Tong, Vinh, Liu, Anji, Hoang, Trung-Dung, Broeck, Guy Van den, Niepert, Mathias
Diffusion Probabilistic Models (DPMs) are powerful generative models showing competitive performance in various domains, including image synthesis and 3D point cloud generation. However, sampling from pre-trained DPMs involves multiple neural function evaluations (NFE) to transform Gaussian noise samples into images, resulting in higher computational costs compared to single-step generative models such as GANs or VAEs. Therefore, a crucial problem is to reduce NFE while preserving generation quality. To this end, we propose LD3, a lightweight framework for learning time discretization while sampling from the diffusion ODE encapsulated by DPMs. LD3 can be combined with various diffusion ODE solvers and consistently improves performance without retraining resource-intensive neural networks. We demonstrate analytically and empirically that LD3 enhances sampling efficiency compared to distillation-based methods, without the extensive computational overhead. We evaluate our method with extensive experiments on 5 datasets, covering unconditional and conditional sampling in both pixel-space and latent-space DPMs. For example, in about 5 minutes of training on a single GPU, our method reduces the FID score from 6.63 to 2.68 on CIFAR10 (7 NFE), and in around 20 minutes, decreases the FID from 8.51 to 5.03 on class-conditional ImageNet-256 (5 NFE). LD3 complements distillation methods, offering a more efficient approach to sampling from pre-trained diffusion models.
Two-view Graph Neural Networks for Knowledge Graph Completion
Tong, Vinh, Nguyen, Dai Quoc, Phung, Dinh, Nguyen, Dat Quoc
To this end, we propose a new KG embedding model, named A knowledge graph (KG) is a network of entity nodes and WGE, to leverage GNNs to capture entity-focused graph structure relationship edges, which can be represented as a collection and relation-focused graph structure for KG completion. of triples in the form of (h, r, t), wherein each triple (h, r, In particular, WGE transforms a given KG into two views. The t) represents a relation r between a head entity h and a tail first view--a single undirected entity-focused graph--only entity t. Here, entities are real-world things or objects such includes entities as nodes to provide the entity neighborhood as music tracks, movies persons, organizations, places and the information. The second view--a single undirected relationfocused like, while each relation type determines a certain relationship graph--considers both entities and relations as nodes, between entities. KGs are used in a number of commercial applications, constructed from constraints (subjective relation, predicate e.g. in such search engines as Google, Microsoft's entity, objective relation), to attain the potential dependence Bing and Facebook's Graph search. They also are useful between two neighborhood relations. Then WGE introduces a resources for many natural language processing tasks such as new encoder module of adopting two vanilla GNNs directly co-reference resolution ([1], [2]), semantic parsing ([3], [4]) on these two graph views to better update entity and relation and question answering ([5], [6]). However, an issue is that embeddings, followed by the decoder module using a weighted KGs are often incomplete, i.e., missing a lot of valid triples score function. In summary, our contributions are as follows: [7].