Entropy rate
In the mathematical theory of probability, the entropy rate or source information rate of a stochastic process is, informally, the time density of the average information in a stochastic process. For stochastic processes with a countable index, the entropy rate H ( X ) {\displaystyle H(X)} is the limit of the joint entropy of n {\displaystyle n} members of the process X k {\displaystyle X_{k}} divided by n {\displaystyle n} , as n {\displaystyle n} tends to infinity: H ( X ) = lim n → ∞ 1 n H ( X 1 , X 2 , … X n ) {\displaystyle H(X)=\lim _{n\to \infty }{\frac {1}{n}}H(X_{1},X_{2},\dots X_{n})} when the limit exists.