Breaking the Curse of Multiagency in Robust Multi-Agent Reinforcement Learning