Causal Convolutional Neural Networks as Finite Impulse Response Filters

Abstract--This study investigates the behavior of Causal Con-volutional Neural Networks (CNNs) with quasi-linear activation functions when applied to time-series data characterized by mul-timodal frequency content. We demonstrate that, once trained, such networks exhibit properties analogous to Finite Impulse Response (FIR) filters, particularly when the convolutional kernels are of extended length exceeding those typically employed in standard CNN architectures. Causal CNNs are shown to capture spectral features both implicitly and explicitly, offering enhanced interpretability for tasks involving dynamic systems. Leveraging the associative property of convolution, we further show that the entire network can be reduced to an equivalent single-layer filter resembling an FIR filter optimized via least-squares criteria. This equivalence yields new insights into the spectral learning behavior of CNNs trained on signals with sparse frequency content. The approach is validated on both simulated beam dynamics and real-world bridge vibration datasets, underlining its relevance for modeling and identifying physical systems governed by dynamic responses. Neural networks have enjoyed wide-spread adoption across various modeling tasks, despite the common pitfall of typically comprising black box models that are often difficult to interpret [1]. It is therefore challenging to tailor a neural network model according to the characteristics of a specific problem: how can we introduce a bias inside a black box? A common way to introduce biases is through the architecture of the neural network. For example, Convolution Neural Networks employ convolutional kernels to force the network to focus on local correlations, which is different from the global connectivity of Multi-Layer Perceptrons. This bias is useful for image processing tasks, where the information of a single pixel is highly correlated with its surrounding pixels [2]. For physics-informed neural networks [3], the bias to be introduced should reflect the prior knowledge on the physical laws that govern the phenomenon that the model is trying to replicate. Due to the black box nature of neural networks, such biases need to be implemented explicitly, e.g. with a physics-informed loss function, rather than an implicit bias in the architecture of the model. In the case of the dynamical behavior of physical systems, a desirable bias should capture the dynamic properties of a system.