Monte Carlo Markov Chain (MCMC) explained

MCMC methods are a family of algorithms that uses Markov Chains to perform Monte Carlo estimate. The name gives us a hint, that it is composed of two components -- Monte Carlo and Markov Chain. Let us understand them separately and in their combined form. Monte Carlo method derives its name from a Monte Carlo casino in Monaco. It is a technique for sampling from a probability distribution and using those samples to approximate desired quantity. In other words, it uses randomness to estimate some deterministic quantity of interest.