PINNACLE: PINN Adaptive ColLocation and Experimental points selection

Physics-Informed Neural Networks (PINNs), which incorporate PDEs as soft constraints, train with a composite loss function that contains multiple training point types: different types of collocation points chosen during training to enforce each PDE and initial/boundary conditions, and experimental points which are usually costly to obtain via experiments or simulations. Training PINNs using this loss function is challenging as it typically requires selecting large numbers of points of different types, each with different training dynamics. PINNACLE uses information on the interaction among training point types, which had not been considered before, based on an analysis of PINN training dynamics via the Neural Tangent Kernel (NTK). We theoretically show that the criterion used by PINNACLE is related to the PINN generalization error, and empirically demonstrate that PINNACLE is able to outperform existing point selection methods for forward, inverse, and transfer learning problems. Deep learning (DL) successes in domains with massive datasets have led to questions on whether it can also be efficiently applied to the scientific domains. In these settings, while training data may be more limited, domain knowledge could compensate by serving as inductive biases for DL training. Such knowledge can take the form of governing Partial Differential Equations (PDEs), which can describe phenomena such as conservation laws or dynamic system evolution in areas such as fluid dynamics (Cai et al., 2021; Chen et al., 2021; Jagtap et al., 2022), wave propagation and optics (bin Waheed et al., 2021; Lin & Chen, 2022), or epidemiology (Rodríguez et al., 2023). Physics-Informed Neural Networks (PINNs) are neural networks that incorporate PDEs and their initial/boundary conditions (IC/BCs) as soft constraints during training (Raissi et al., 2019), and have been successfully applied to various problems. These include forward problems (i.e., predicting PDE solutions given specified PDEs and ICs/BCs) and inverse problems (i.e., learning unknown PDE parameters given experimental data). However, the training of PINNs is challenging. Past works have tried to separately address these individually by considering an adaptive selection of CL points (Nabian et al., 2021; Gao & Wang, 2023; Wu et al., 2023; Peng et al., 2022; Zeng et al., 2022; Tang et al., 2023), or E Some works have also proposed heuristics that adjust loss term weights of the various point types to try improve training dynamics, but do not consider point selection (Wang et al., 2022c). However, no work thus far has looked into optimizing all training point types jointly to significantly boost PINN performance. Given that the solution spaces of the PDE, IC/BC and underlying output function are tightly coupled, it is inefficient and sub-optimal to select each type of training points separately and ignore cross information across point types.