Multi-agent learning under uncertainty: Recurrence vs. concentration