Belief States for Cooperative Multi-Agent Reinforcement Learning under Partial Observability