Performant, Memory Efficient and Scalable Multi-Agent Reinforcement Learning