A new study has found that Fourier analysis, a mathematical technique that has been around for 200 years, can be used to reveal important information about how deep neural networks learn to perform complex physics tasks, such as climate and turbulence modeling. This research highlights the potential of Fourier analysis as a tool for gaining insights into the inner workings of artificial intelligence and could have significant implications for the development of more effective machine learning algorithms. Fourier transformations reveal how deep neural network learns complex physics. One of the oldest tools in computational physics -- a 200-year-old mathematical technique known as Fourier analysis -- can reveal crucial information about how a form of artificial intelligence called a deep neural network learns to perform tasks involving complex physics like climate and turbulence modeling, according to a new study. The discovery by mechanical engineering researchers at Rice University is described in an open-access study published in the journal PNAS Nexus, a sister publication of the Proceedings of the National Academy of Sciences.

We sought to investigate whether artificial intelligence (AI) and specifically deep neural networks for electrocardiogram (ECG) signal analysis can be explained using human selected features. We also sought to quantify such explainability and test if the AI model learns features that are similar to a human expert. We used a set of 100,000 ECGs that were annotated by human explainable features. We applied both linear and nonlinear models to predict published ECG AI models output for the detection of patients' age and sex. We further used canonical correlation analysis to quantify the amount of shared information between the features found in an unsupervised way.

We develop a variational framework to understand the properties of functions learned by deep neural networks with ReLU activation functions fit to data. We propose a new function space, which is reminiscent of classical bounded variation spaces, that captures the compositional structure associated with deep neural networks. We derive a representer theorem showing that deep ReLU networks are solutions to regularized data fitting problems in this function space. The function space consists of compositions of functions from the (non-reflexive) Banach spaces of second-order bounded variation in the Radon domain. These are Banach spaces with sparsity-promoting norms, giving insight into the role of sparsity in deep neural networks. The neural network solutions have skip connections and rank bounded weight matrices, providing new theoretical support for these common architectural choices. The variational problem we study can be recast as a finite-dimensional neural network training problem with regularization schemes related to the notions of weight decay and path-norm regularization. Finally, our analysis builds on techniques from variational spline theory, providing new connections between deep neural networks and splines.