Algorithmically Designed Artificial Neural Networks (ADANNs): Higher order deep operator learning for parametric partial differential equations

Deep learning approximation methods - usually consisting of deep artificial neural networks (ANN) trained through stochastic gradient descent (SGD) optimization methods - belong nowadays to the most heavily employed approximation methods in the digital world. The striking feature of deep learning methods is that in many situations numerical simulations suggest that the computational effort of such methods seem to grow only at most polynomially in the input dimension d N = {1, 2, 3,... } of the problem under consideration. In contrast, classical numerical methods usually suffer under the so-called curse of dimensionality (cf., e.g., Bellman [4], Novak & Wozniakowski [37, Chapter 1], and Novak & Wozniakowski [38, Chapter 9]) in the sense that the computational effort grows at least exponentially in the dimension. In the recent years, deep learning technologies have also been intensively used to attack problems from scientific computing such as the numerical solutions of partial differential equations (PDEs). In particular, deep learning approximation methods have been used to approximately solve high-dimensional nonlinear PDEs (see, e.g., [2,5,10,11,14,16,25,42] and the references mentioned therein) such as high-dimensional nonlinear pricing problems from financial engineering and Hamiltonian-Jacobi-Bellman equations from optimal control. In the context of such highdimensional nonlinear PDEs, the progress of deep learning approximation methods is obvious as there are - except of in some special cases (see, e.g., [19, 20, 36] and the references therein for Branching type methods and see, e.g., [11-13, 22] and the references therein for multilevel Picard methods) - essentially no alternative numerical approximation methods which are capable of solving such high-dimensional nonlinear PDEs. There is nowadays also a huge literature on deep learning approximation methods for lowdimensional PDEs (cf., e.g., [24, 41]).