Deep transfer operator learning for partial differential equations under conditional shift

These authors contributed equally to this work. Abstract Transfer learning (TL) enables the transfer of knowledge gained in learning to perform one task (source) to a related but different task (target), hence addressing the expense of data acquisition and labeling, potential computational power limitations, and dataset distribution mismatches. We propose a new TL framework for task-specific learning (functional regression in partial differential equations (PDEs)) under conditional shift based on the deep operator network (DeepONet). Task-specific operator learning is accomplished by fine-tuning task-specific layers of the target DeepONet using a hybrid loss function that allows for the matching of individual target samples while also preserving the global properties of the conditional distribution of target data. Inspired by the conditional embedding operator theory, we minimize the statistical distance between labeled target data and the surrogate prediction on unlabeled target data by embedding conditional distributions onto a reproducing kernel Hilbert space. We demonstrate the advantages of our approach for various TL scenarios involving nonlinear PDEs under diverse conditions due to shift in the geometric domain and model dynamics. TL framework enables fast and efficient learning of heterogeneous tasks despite significant differences between the source and target domains. Deep learning has been successfully employed to simulate computationally expensive complex physical processes described by partial differential equations (PDEs) and achieve superior performance that allows the acceleration of numerous tasks including uncertainty quantification (UQ), risk modeling and design optimization [1-6]. Despite this success, the predictive performance of such models is often limited by the availability of labeled data used for training. However, in many cases collecting large and sufficient labeled datasets can be computationally intractable (e.g., when high-fidelity or multiscale models are considered). Furthermore, learning in isolation, i.e., training a single predictive model for different but related tasks, can be extremely expensive. To tackle this bottleneck, knowledge between relevant domains can be leveraged in a framework known as transfer learning (TL) [7]. In this scenario, information from a model trained on a specific domain (source) with sufficient labeled data can be transferred to a different but closely related domain (target) for which only a small number of training data is available.