Partially Observable Multi-agent RL with (Quasi-)Efficiency: The Blessing of Information Sharing