Is $L^2$ Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?