MuZero

We refer to the paper Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model and to this official DeepMind pseudocode for the coding part. The MuZero algorithm combines a tree-based search with a learned model, achieving superhuman performance in a variety of visually complex domains, without any knowledge of their underlying dynamics. We will give a brief explanation of how MuZero works, using diagrams as intuitive support. MuZero is an algorithm for mastering games like Go, Chess, Shogi and Atari without explicitly knowing the rules, thanks to its ability to plan winning strategies in unspecified environments. Trying to overcome the limitations of previous algorithms like AlphaZero, MuZero does not model the entire environment, it just models aspects that are crucial to the agent's decision-making process: the value (how good is the current position), the policy (which action is the best to take) and the reward (how good was last action).