Scalable Offline Reinforcement Learning for Mean Field Games