ODEFormer: Symbolic Regression of Dynamical Systems with Transformers

Recent triumphs of machine learning (ML) spark growing enthusiasm for accelerating scientific discovery [1-3]. In particular, inferring dynamical laws governing observational data is an extremely challenging task that is anticipated to benefit substantially from modern ML methods. Modeling dynamical systems for forecasting, control, and system identification has been studied by various communities within ML. Successful modern approaches are primarily based on advances in deep learning, such as neural ordinary differential equation (NODE) (see Chen et al. [4] and many extensions thereof). However, these models typically lack interpretability due to their black-box nature, which has inspired extensive research on explainable ML of overparameterized models [5, 6].