Improving physics-informed DeepONets with hard constraints

Recent years have seen tremendous interest in solving differential equations with neural networks. Originally introduced in [7] and popularized through [12], in which it is referred to as the method physics-informed neural networks, it has become popular throughout the mathematical sciences, with applications to astronomy [11], biomedical engineering [8], geophysics [13] and meteorology [3], just to name a few. While the underlying method is conceptually straightforward to implement, several failure modes of the original method have been identified in the past [2, 6, 15, 16, 18], along with some mitigation strategies. Setting these training difficulties aside, another fundamental shortcoming of physics-informed neural networks is that they require extensive training, to the point where they are seldom computationally competitive compared to standard numerical methods, see [4] for an example related to weather prediction. The main issue is that each changing of initial and/or boundary conditions for a system of differential equations requires retraining of the neural network solution approximator, which is a costly endeavour, especially when accurate solutions are required. One strategy to overcome this issue is to not learn the solution of a differential equation itself, but rather the solution operator. This idea, relying on the universal approximation theorem for operators [5], was proposed in [10], where it is referred to as physics-informed deep operator approach, or physics-informed DeepONet.