5 Ways NOT to Build a Catan AI

During the pandemic, I started playing online Settlers of Catan (shoutout to https://colonist.io!). I quickly realized there is more skill involved than one may think and this made the game beautiful for me. At the same time, I was amazed at the recent success of the AlphaGo team at making a superhuman player in the games of Chess, Shogi, and Go, with a seemingly simple algorithm. These two interests prompted me to take a shot at making a superhuman artificial intelligence player for Catan. The purpose of this post is to share these attempts so that others can take these findings further. There were two main ideas I wanted to explore: Reinforcement Learning (like AlphaZero) and to use Supervised Learning to build a data-driven "value function" (a function that tells us how good the position of a given player is).