VS-PINN: A fast and efficient training of physics-informed neural networks using variable-scaling methods for solving PDEs with stiff behavior

Bridging the gap between classical scientific computing and machine learning, the emerging field called scientific machine learning has introduced a completely different framework to compute the solutions of partial differential equations (PDEs). The forefront of this evolution lies within the area of physicsinformed neural networks (PINNs) [18, 10]. Based on the universal approximation property of deep neural networks, the PINNs approximate the solutions of PDEs by incorporating underlying physics into deep neural networks. Due to their simplicity and flexibility in handling a wide range of physical problems involving PDEs, PINNs have recently gained great attention and have been applied to various fields in computational science: bio-medical science [22, 11], fluids mechanics [20, 19, 9, 4, 21], uncertainty quantification [30, 32, 29] and meta-material design [12, 3]. Moreover, since the PINNs utilize randomlyselected collocation points as training samples in the spatio-temporal domain, the PINNs are available for high-dimensional PDEs [24, 6], on the domains with complex geometries [13, 23, 26, 14]. However, despite the significant empirical success of PINNs, we only have limited knowledge about the behavior of these constrained neural networks during their training process, and the training of PINNs often fails. In particular, since neural networks typically assume a smooth prior, it is often challenging to train PINNs to learn a solution with a sharp transition which poses significant obstacles for the model prediction. For example, some recent studies have illustrated that training PINNs with fully-connected neural networks usually suffers from so-called spectral bias or F-principle meaning that it is difficult for PINNs to learn functions with high frequencies [17, 2, 25].