Goto

Collaborating Authors

 information flow






Appendix

Neural Information Processing Systems

As shown in Figure 6, node-viewGTransformer contains nodedyMPN, which maintains hidden states of nodeshv,v V and performs the message passing over nodes.




CanInformationFlowsSuggestTargets Appendices

Neural Information Processing Systems

Note how the most dominant accuracy flows arise fromX3, which is the only bias-free feature in the dataset. In contrast, the largest bias flows arise fromX1 and X2, both of which are heavily biased features.



On the Practical Estimation and Interpretation of Rényi Transfer Entropy

Tabachová, Zlata, Jizba, Petr, Lavička, Hynek, Paluš, Milan

arXiv.org Machine Learning

Rényi transfer entropy (RTE) is a generalization of classical transfer entropy that replaces Shannon's entropy with Rényi's information measure. This, in turn, introduces a new tunable parameter $α$, which accounts for sensitivity to low- or high-probability events. Although RTE shows strong potential for analyzing causal relations in complex, non-Gaussian systems, its practical use is limited, primarily due to challenges related to its accurate estimation and interpretation. These difficulties are especially pronounced when working with finite, high-dimensional, or heterogeneous datasets. In this paper, we systematically study the performance of a k-nearest neighbor estimator for both Rényi entropy (RE) and RTE using various synthetic data sets with clear cause-and-effect relationships inherent to their construction. We test the estimator across a broad range of parameters, including sample size, dimensionality, memory length, and Rényi order $α$. In particular, we apply the estimator to a set of simulated processes with increasing structural complexity, ranging from linear dynamics to nonlinear systems with multi-source couplings. To address interpretational challenges arising from potentially negative RE and RTE values, we introduce three reliability conditions and formulate practical guidelines for tuning the estimator parameters. We show that when the reliability conditions are met and the parameters are calibrated accordingly, the resulting effective RTE estimates accurately capture directional information flow across a broad range of scenarios. Results obtained show that the explanatory power of RTE depends sensitively on the choice of the Rényi parameter $α$. This highlights the usefulness of the RTE framework for identifying the drivers of extreme behavior in complex systems.