A Markov chain is a mathematical system that undergoes transitions from one state to another on a state space.

A Markov chain is a random process usually characterized as memoryless: the next state depends only on the current state and not on the sequence of events that preceded it. This specific kind of "memorylessness" is called the Markov property. Markov chains have many applications as statistical models of real-world processes. Markov chains are in discete time (aka, a discrete-time Markov chain or DTMC), however in Probability Theory, Markov chains are in continuos-time, aka CTMC or continuous-time Markov Process.

history | excerpt history