Frequency-adaptive tensor neural networks for high-dimensional multi-scale problems Jizu Huang, Rukang Y ou, T ao Zhou The training dynamics of T ensor Neural Networks (TNNs) are shown to be influenced by the Frequency Principle, as revealed by a Fourier-based analysis. W e improve the expressivity of TNNs for high-dimensional multi-scale problems by integrating random Fourier features. W e develop a frequency-adaptive TNNs algorithm that e fficiently extracts frequency features of high-dimensional functions by leveraging the intrinsic tensor structure. Abstract T ensor neural networks (TNNs) have demonstrated their superiority in solving high-dimensional problems. However, similar to conventional neural networks, TNNs are also influenced by the Frequency Principle, which limits their ability to accurately capture high-frequency features of the solution. In this work, we analyze the training dynamics of TNNs by Fourier analysis and enhance their expressivity for high-dimensional multi-scale problems by incorporating random Fourier features. Leveraging the inherent tensor structure of TNNs, we further propose a novel approach to extract frequency features of high-dimensional functions by performing the Discrete Fourier T ransform to one-dimensional component functions. Building on this idea, we propose a frequency-adaptive TNNs algorithm, which significantly improves the ability of TNNs in solving complex multi-scale problems. Extensive numerical experiments are performed to validate the e ffectiveness and robustness of the proposed frequency-adaptive TNNs algorithm. Introduction Building upon their groundbreaking achievements in computer vision [1], speech recognition [2], and natural language processing [3-5], deep neural networks (DNNs) have emerged as a promising paradigm for scientific computing, particularly in solving partial di fferential equations (PDEs) [6-16].

Physics-informed neural networks (PINNs) have been increasingly employed due to their capability of modeling complex physics systems. To achieve better expressiveness, increasingly larger network sizes are required in many problems. This has caused challenges when we need to train PINNs on edge devices with limited memory, computing and energy resources. To enable training PINNs on edge devices, this paper proposes an end-to-end compressed PINN based on Tensor-Train decomposition. In solving a Helmholtz equation, our proposed model significantly outperforms the original PINNs with few parameters and achieves satisfactory prediction with up to 15$\times$ overall parameter reduction.