Remembering the Markov Property in Cooperative MARL