Orthogonal greedy algorithm for linear operator learning with shallow neural network

Greedy algorithms, particularly the orthogonal greedy algorithm (OGA), have proven e ff ective in training shallow neural networks for fitting functions and solving partial di fferential equations (PDEs). In this paper, we extend the application of OGA to the tasks of linear operator learning, which is equivalent to learning the kernel function through integral transforms. Firstly, a novel greedy algorithm is developed for kernel estimation rate in a new semi-inner product, which can be utilized to approximate the Green's function of linear PDEs from data. Secondly, we introduce the OGA for point-wise kernel estimation to further improve the approximation rate, achieving orders of accuracy improvement across various tasks and baseline models. In addition, we provide a theoretical analysis on the kernel estimation problem and the optimal approximation rates for both algorithms, establishing their e fficacy and potential for future applications in PDEs and operator learning tasks. Introduction In recent years, deep neural networks have emerged as a powerful tool for solving partial di ff erential equations (PDEs) in a wide range of scientific and engineering domains [1]. Approaches in this area can be broadly classified into two main categories: (1) single PDE solvers and (2) operator learning. Single PDE solvers, such as physics-informed neural networks(PINNs)[2], the deep Galerkin method[3], the deep Ritz method[4], optimize the deep neural network by minimizing a given loss function related to the PDE. These methods are specifically designed to solve a given instance of the PDE. In contrast, operator learning involves using deep neural networks to learn operators between function spaces, allowing for the learning of solution operators of PDEs from data pairs. Recently, several operator learning methods have been proposed, including deep Green networks (DGN)[5], deep operator networks (DON)[6], and neural operators (NOs)[7].