Markov Chain - AI Summary

Each state has a certain probability of transitioning to each other state, so each time you are in a state and want to transition, a markov chain can predict outcomes based on pre-existing probability data. A Markov model is a stochastic model with the property that future states are determined only by the current state -- in other words, the model has no memory; it only knows what state it's in now, not any of the states which occurred previously. A Markov chain is one example of a Markov model, but other examples exist. One other example commonly used in the field of artificial intelligence is the Hidden Markov model, which is a Markov chain for which the state is not directly observable. There are quite a few applications of Markov Chains to AI -- Markov Chains are useful basically when you want to model something that's in discrete states, but you don't understand how it works.