From Finite to Countable-Armed Bandits