Learning to Cooperate, Compete, and Communicate

Multiagent environments where agents compete for resources are stepping stones on the path to AGI. Multiagent environments have two useful properties: first, there is a natural curriculum -- the difficulty of the environment is determined by the skill of your competitors (and if you're competing against clones of yourself, the environment exactly matches your skill level). Second, a multiagent environment has no stable equilibrium: no matter how smart an agent is, there's always pressure to get smarter. These environments have a very different feel from traditional environments, and it'll take a lot more research before we become good at them. We've developed a new algorithm, MADDPG, for centralized learning and decentralized execution in multiagent environments, allowing agents to learn to collaborate and compete with each other.