Transfer entropy
Transfer entropy is a non-parametric statistic measuring the amount of directed (time-asymmetric) transfer of information between two random processes. Transfer entropy from a process X to another process Y is the amount of uncertainty reduced in future values of Y by knowing the past values of X given past values of Y. More specifically, if X t {\displaystyle X_{t}} and Y t {\displaystyle Y_{t}} for t ∈ N {\displaystyle t\in \mathbb {N} } denote two random processes and the amount of information is measured using Shannon's entropy, the transfer entropy can be written as: T X → Y = H ( Y t ∣ Y t − 1 : t − L ) − H ( Y t ∣ Y t − 1 : t − L , X t − 1 : t − L ) , {\displaystyle T_{X\rightarrow Y}=H\left(Y_{t}\mid Y_{t-1:t-L}\right)-H\left(Y_{t}\mid Y_{t-1:t-L},X_{t-1:t-L}\right),} where H(X) is Shannon's entropy of X. The above definition of transfer entropy has been extended by other types of entropy measures such as Rényi entropy.