Explain Like I'm Five: Using LLMs to Improve PDE Surrogate Models with Text

Solving Partial Differential Equations (PDEs) is ubiquitous in science and engineering. Computational complexity and difficulty in writing numerical solvers has motivated the development of machine learning techniques to generate solutions quickly. Many existing methods are purely data driven, relying solely on numerical solution fields, rather than known system information such as boundary conditions and governing equations. However, the recent rise in popularity of Large Language Models (LLMs) has enabled easy integration of text in multimodal machine learning models. In this work, we use pretrained LLMs to integrate various amounts known system information into PDE learning. Our multimodal approach significantly outperforms our baseline model, FactFormer, in both next-step prediction and autoregressive rollout performance on the 2D Heat, Burgers, Navier-Stokes, and Shallow Water equations. Further analysis shows that pretrained LLMs provide highly structured latent space that is consistent with the amount of system information provided through text. Solving Partial Differential Equations (PDEs) is the cornerstone of many areas of science and engineering, from quantum mechanics to fluid dynamics. While traditional numerical solvers often have rigorous error bounds, they are limited in scope, where different methods are required for different governing equations, and different regimes even for a single governing equation. In the area of fluid dynamics, especially, solvers that are designed for Navier Stokes equations generally will not perform optimally in both the laminar and turbulent flow regimes.