Efficient Multi-agent Reinforcement Learning by Planning