The Markov Property

The Markov property Pr (X_(n+1)=x ?| X_1= x_1,X_2= x_2,.,X_n= x_n)= Pr (X_(n+1)=x ?| X_n= x_n), Pr (X_1= x_1,X_2= x_2,.,X_n= x_n)>0

Pr (Xn+1 = x | X1 = x1, X2 = x2, ., Xn = xn, ) = Pr ( Xn+1 = x | Xn = xn ) , Pr ( X1 = x1, X2 = x2, ., Xn = xn ) > 0, i.e. both conditional probabilities are 'well defined'. A sequence of random variables X1, X2, X3, ... is said to have the Markov property if each variable Xn+1 in the sequence is dependent only on the variable Xn preceding it and is independent of those variables in the sequence prior to that, i.e. Xn-1, Xn-2, ., X1. This property of a stochastic sequence is sometimes loosely alluded to as 'memorylessness'. A sequence characterised by the Markov property is referred to as a Markov chain. The term Markov process can also be used, though the latter is often reserved for a discrete-time Markov chain (DTMC). Markov processes are named after the Russian Mathematician Andrey Markov (1856-1922) and have many applications as statistical models of real-world processes. The possible values of Xn form a countable set called the state space of the chain. A Markov chain is often described by a state transition diagram, specifically a directed graph, where the state transitions are labelled by the probabilities of going from one state at instance n to the other states at instance n+1. The same information can be represented as a state transition matrix. A state transition diagram for a simple example is illustrated. The states represent whether a hypothetical stock market is exhibiting a bull market, bear market, or mixed market trend during a given time period. According to the figure, a bull week is followed by another bull week 90% of the time, a bear week 7.5% of the time, and a stagnant week the other 2.5% of the time. Markovian systems appear extensively in the fields of thermodynamics and statistical mechanics. In chemistry they can be used to model enzyme activity, copolymers and some cases of crystallisation. They are used in information sciences and queuing theory. The PageRank of a webpage as used by Google is apparently defined by a Markov chain (at the time of writing). They are applied in biology, social sciences, games and music, inter alia.