A Law of Iterated Logarithm for Multi-Agent Reinforcement Learning