A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning