Learning in Feedforward Neural Networks Accelerated by Transfer Entropy

We generally differentiate between statistical correlation and causality. Often, when correlation is observed, causality is wrongly inferred and we are tempted to identify causality through correlation. This is because of the inability to detect a time lag between a cause and effect, which is a prerequisite for causality [1]. Following Shadish et al. [2], the three key criteria for inferring a cause and effect relationship are (1) the cause preceded the effect, (2) the cause was related to the effect, and (3) we can find no plausible alternative explanation for the effect other than the cause. According to [3], there is an important distinction between the "intervention-based causality" and "statistical causality". The first concept, introduced by Pearl [4], combines statistical and non-statistical data and allows one to answer questions, like "if we give a drug to a patient, i.e., intervene, will their chances of survival increase?" Statistical causality does not answer such questions, because it does not operate on the concept of intervention and only involves tools of data analysis. The causality in a statistical sense is a type of dependence, where we infer direction as a result of the knowledge of temporal structure and the notion that the cause has to precede the effect. We will focus here only on statistical causality measured by the information transfer approach.