Reviews: Thompson Sampling and Approximate Inference

Thompson Sampling with Approximate Inference This paper investigates the performance of Thompson sampling, when the sampled distribution does not match the "problem" distribution exactly. The authors clearly explain some settings where mismatched sampling distributions can cause linear regret. The authors support their analysis with some expository "toy" experiments. There are several things to like about this paper: - This paper is one of the first to provide a clear analysis of Thompson sampling in the regime of imperfect inference. It seems like another solution that would "intuitively work" is to artificially expand the "prior" of the Thompson sampling procedure, but in a way that would concentrate away with data.