Scaling up Deep Learning for PDE-based Models

Solving partial differential equations (PDEs) underlies much of applied mathematics and engineering, ranging from computer graphics and financial pricing, to civil engineering and weather prediction. Conventional approaches to prediction in PDE models rely on numerical solvers and require substantial computing resources in the model-application phase. While in some application domains, such as structural engineering, the longer run-times may be acceptable, in domains with rapid decay of value of the prediction, such as weather forecasting, the run-time of the solver is of paramount importance. In many such applications, the ability to generate large volumes of data facilitates the use of surrogate or reduced-order models [1] obtained using deep artificial neural networks [2]. Although the observation that artificial neural networks could be applied to physical models is not new [3, 4, 5, 6, 7, 8, 9, 5, 10], and indeed, it is seen as one of the key trends [11, 12, 13] on the interface of applied mathematics, data science, and deep learning, their applications did not reach the level of success observed in the field of the image classification, speech recognition, machine translation, and other problems processing unstructured high-dimensional data, yet. A key issue faced by applications of deep-learning techniques to physical models is their scalability. Even very recent research on deep-learning for physical models [14, 15, 16] uses a solver for PDEs to obtain hundreds of thousands of outputs. The deep learning can then be seen as means of nonlinear regression between the inputs and outputs.