Distributed Exploration in Multi-Armed Bandits

We study exploration in Multi-Armed Bandits in a setting where k players collaborate inorder to identify an ε-optimal arm. Our motivation comes from recent employment of bandit algorithms in computationally intensive, large-scale applications. Ourresults demonstrate a nontrivial tradeoff between the number of arm pulls required by each of the players, and the amount of communication between them. In particular, our main result shows that by allowing the k players to communicate onlyonce, they are able to learn k times faster than a single player. That is, distributing learning to k players gives rise to a factor k parallel speedup. We complement this result with a lower bound showing this is in general the best possible. On the other extreme, we present an algorithm that achieves the ideal factor k speedup in learning performance, with communication only logarithmic in1/ε.