Practical Aspects on Solving Differential Equations Using Deep Learning: A Primer

Deep learning (DL) [44] has advanced significantly in recent years, providing algorithms that can solve complex problems varying from image classification [12] to playing games such as Go [59] at a human level or even better. More recent advances include the development of Large Language Models (LLMs), which are substantial deep learning models (i.e., they usually have billions of parameters) such as ChatGPT [63, 21], and Llama [66] that have been trained on enormous data sets. Deep learning has naturally affected many scientific fields and has been applied to many complex problems. One such problem is the numerical solution of differential equations, where (deep) neural networks approximate the solution of partial or ordinary differential equations. The idea is not new since there are works from the 90s like Lee et al. [45] and Wang et al. [67] have relied on Hopfield neural networks [33] to solve differential equations using linear systems after discretizing them. Meade and Fernandez have proposed solvers for nonlinear differential equations based on combining neural networks and splines [48]. Lagaris et al. introduced in [42] an efficient method that relies on trial solutions and neural networks to solve boundary and initial values problems. They form a loss function using the differential operator of the problem, and a neural network approximates the solution by minimizing that loss function.