A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks