Text2PDE: Latent Diffusion Models for Accessible Physics Simulation

Recent advances in deep learning have inspired numerous works on data-driven solutions to partial differential equation (PDE) problems. These neural PDE solvers can often be much faster than their numerical counterparts; however, each presents its unique limitations and generally balances training cost, numerical accuracy, and ease of applicability to different problem setups. To address these limitations, we introduce several methods to apply latent diffusion models to physics simulation. Firstly, we introduce a mesh autoencoder to compress arbitrarily discretized PDE data, allowing for efficient diffusion training across various physics. Furthermore, we investigate full spatio-temporal solution generation to mitigate autoregressive error accumulation. Lastly, we investigate conditioning on initial physical quantities, as well as conditioning solely on a text prompt to introduce text2PDE generation. We show that language can be a compact, interpretable, and accurate modality for generating physics simulations, paving the way for more usable and accessible PDE solvers. Through experiments on both uniform and structured grids, we show that the proposed approach is competitive with current neural PDE solvers in both accuracy and efficiency, with promising scaling behavior up to 3 billion parameters. By introducing a scalable, accurate, and usable physics simulator, we hope to bring neural PDE solvers closer to practical use. Neural PDE solvers are an exciting new class of physics solvers that have the potential to improve many aspects of conventional numerical solvers. Initial works have proposed various architectures to accomplish this, such as graph-based (Li & Farimani, 2022; Battaglia et al., 2016), physicsinformed (Raissi et al., 2019), or convolutional approaches (Thuerey et al., 2020). Subsequent work on developing neural operators (Li et al., 2021; Lu et al., 2021; Kovachki et al., 2023) established them as powerful physics approximators that can quickly and accurately predict PDEs, and recent work has focused on improving many of their different aspects. Despite these advances, there are still many factors that limit the practical adoption of neural PDE surrogates. Figure 1: We introduce latent diffusion models for physics simulation, with the remarkable ability of generating an entire PDE rollout from a text prompt. Two generated solutions are displayed with their model inputs.