Are Statistical Methods Obsolete in the Era of Deep Learning?

The advancement of deep neural network models in the last fifteen years has profoundly altered the scientific landscape of estimation, prediction and decision making, from the early success of image recognition (Krizhevsky et al., 2012; He et al., 2016), to the success of self-learning of board games (Silver et al., 2017), to machine translation (Wu et al., 2016), to generative AI (Ho et al., 2020), and to the success of protein structure prediction (Jumper et al., 2021), among many other developments. In many of these successes, there are no well-established mechanistic models to describe the underlying problem (for example, we do not fully understand how human brains translate from one language to another). As such, it is conceivable that such successes are attributable to deep neural networks' remarkable capabilities for universal function approximation. In contrast, the hand-crafted models that existed before deep neural networks (such as n-gram models (Katz, 1987; Brown et al., 1992; Bengio et al., 2000)) were too restricted to offer satisfactory approximation. How well do deep neural network models work when there are well-established mechanistic models (as in physical sciences, where decades of theoretical and experimental endeavor have yielded highly accurate mechanistic models in many cases) -- in particular, how do the inference and prediction results of deep neural network models compare to more statistical approaches in the presence of reliable mechanistic models -- is an interesting question.