Appendix Gigastep - One Billion Steps per Second Multi-agent Reinforcement Learning