Identifying Information-Transfer Nodes in a Recurrent Neural Network Reveals Dynamic Representations

Abstract--Understanding the internal dynamics of Recurrent Neural Networks (RNNs) is crucial for advancing their inter-pretability and improving their design. This study introduces an innovative information-theoretic method to identify and analyze information-transfer nodes within RNNs, which we refer to as information relays. We apply this methodology to both synthetic and real-world time series classification tasks, employing various RNN architectures, including Long Short-T erm Memory (LSTM) networks and Gated Recurrent Units (GRUs). Our results reveal distinct patterns of information relay across different architectures, offering insights into how information is processed and maintained over time. Additionally, we conduct node knockout experiments to assess the functional importance of identified nodes, significantly contributing to explainable artificial intelligence by elucidating how specific nodes influence overall network behavior . This study not only enhances our understanding of the complex mechanisms driving RNNs but also provides a valuable tool for designing more robust and interpretable neural networks. RTIFICIAL neural networks represent a significant enigma. On the one hand, we grasp the mechanism of backpropagation and the dataset that optimally trains them; on the other hand, we do not fully comprehend how ANNs or their recurrent counterparts, RNNs, accomplish their tasks. At their most basic, their function can be described as follows: input vectors are propagated forward through multiple layers by applying a dot product between matrices and these vectors, alongside the application of mathematical threshold functions to the vectors. While this functional description enables us to implement and compute neural networks relatively easily, it applies to a network executing the MNIST handwritten numeral task or the CIFAR-10 image classification problem. The functional differences between these tasks lie in the variations between their weight matrices, which yield complex hidden state vectors that contain and propagate the processed information. Numerous efforts have been made to decode the roles of these vector components.