Almost Free Exploration from Decentralized Learning Agents