Attention (as Discrete-Time Markov) Chains

Neural Information Processing Systems 

We introduce a new interpretation of the attention matrix as a discrete-time Markov chain.