Local Bandit Approximation for Optimal Learning Problems