Frequency-adaptive tensor neural networks for high-dimensional multi-scale problems

Frequency-adaptive tensor neural networks for high-dimensional multi-scale problems Jizu Huang, Rukang Y ou, T ao Zhou The training dynamics of T ensor Neural Networks (TNNs) are shown to be influenced by the Frequency Principle, as revealed by a Fourier-based analysis. W e improve the expressivity of TNNs for high-dimensional multi-scale problems by integrating random Fourier features. W e develop a frequency-adaptive TNNs algorithm that e fficiently extracts frequency features of high-dimensional functions by leveraging the intrinsic tensor structure. Abstract T ensor neural networks (TNNs) have demonstrated their superiority in solving high-dimensional problems. However, similar to conventional neural networks, TNNs are also influenced by the Frequency Principle, which limits their ability to accurately capture high-frequency features of the solution. In this work, we analyze the training dynamics of TNNs by Fourier analysis and enhance their expressivity for high-dimensional multi-scale problems by incorporating random Fourier features. Leveraging the inherent tensor structure of TNNs, we further propose a novel approach to extract frequency features of high-dimensional functions by performing the Discrete Fourier T ransform to one-dimensional component functions. Building on this idea, we propose a frequency-adaptive TNNs algorithm, which significantly improves the ability of TNNs in solving complex multi-scale problems. Extensive numerical experiments are performed to validate the e ffectiveness and robustness of the proposed frequency-adaptive TNNs algorithm. Introduction Building upon their groundbreaking achievements in computer vision [1], speech recognition [2], and natural language processing [3-5], deep neural networks (DNNs) have emerged as a promising paradigm for scientific computing, particularly in solving partial di fferential equations (PDEs) [6-16].