Additive Markov chain

Definition

An additive Markov chain of order m is a sequence of random variables X1, X2, X3, ..., possessing the following property: the probability that a random variable X**n has a certain value x**n under the condition that the values of all previous variables are fixed depends on the values of m previous variables only (Markov chain of order m), and the influence of previous variables on a generated one is additive,

:\Pr(X_n=x_n\mid X_{n-1}=x_{n-1}, X_{n-2}=x_{n-2}, \dots, X_{n-m}=x_{n-m}) = \sum_{r=1}^{m} f(x_n,x_{n-r},r).

Binary case

A binary additive Markov chain is where the state space of the chain consists on two values only, Xn ∈ { x1, x2 }. For example, X**n ∈ { 0, 1 }. The conditional probability function of a binary additive Markov chain can be represented as

: \Pr(X_n=1\mid X_{n-1}=x_{n-1}, X_{n-2}=x_{n-2}, \dots) = \bar{X} + \sum_{r=1}^{m} F(r) (x_{n-r}-\bar{X}),

: \Pr(X_n=0\mid X_{n-1}=x_{n-1}, X_{n-2}=x_{n-2}, \dots) = 1 - \Pr(X_n=1\mid X_{n-1} = x_{n-1}, X_{n-2} = x_{n-2}, \dots).

Here \bar{X} is the probability to find X**n = 1 in the sequence and F(r) is referred to as the memory function. The value of \bar{X} and the function F(r) contain all the information about correlation properties of the Markov chain.

Relation between the memory function and the correlation function

In the binary case, the correlation function between the variables X_n and X_k of the chain depends on the distance n - k only. It is defined as follows: :K(r) = \langle (X_n-\bar{X})(X_{n+r}-\bar{X}) \rangle = \langle X_n X_{n+r} \rangle -{\bar{X}}^2,

where the symbol \langle \cdots \rangle denotes averaging over all n. By definition,

:K(-r)=K(r), K(0)=\bar{X}(1-\bar{X}).

There is a relation between the memory function and the correlation function of the binary additive Markov chain:S.S. Melnyk, O.V. Usatenko, and V.A. Yampol'skii. (2006) "Memory functions of the additive Markov chains: applications to complex dynamic systems", Physica A, 361 (2), 405–415

:K(r)=\sum_{s=1}^m K(r-s)F(s), , , , , r=1, 2, \dots,.

Notes

References

• A.A. Markov. (1906) "Rasprostranenie zakona bol'shih chisel na velichiny, zavisyaschie drug ot druga". Izvestiya Fiziko-matematicheskogo obschestva pri Kazanskom universitete, 2-ya seriya, tom 15, 135–156
• A.A. Markov. (1971) "Extension of the limit theorems of probability theory to a sum of variables connected in a chain". reprinted in Appendix B of: R. Howard. Dynamic Probabilistic Systems, volume 1: Markov Chains. John Wiley and Sons
• Ramakrishnan, S. (1981) "Finitely Additive Markov Chains", Transactions of the American Mathematical Society, 265 (1), 247–272