Less Is More: Robust Robot Learning via Partially Observable Multi-Agent Reinforcement Learning