A Framework for Scalable Heterogeneous Multi-Agent Adversarial Reinforcement Learning in IsaacLab

Research on adversarial reinforcement learning has developed along several trajectories, from early demonstrations of self-play to large-scale competitive frameworks and physics-based multi-agent domains. A. Early Adversarial Self-Play One of the first demonstrations of emergent competition in physics-based environments introduced competitive tasks in MuJoCo [9], [12]. They demonstrated that self-play can naturally induce curricula, with agents developing increasingly complex behaviors. Extension of this work [13] highlighted new aspects of adversarial training that exploited brittle policies, those which appeared robust under standard evaluation, to improve agent performance. B. Multi-Agent Extensions As adversarial learning moved toward multi-agent settings, algorithmic advances became central.