Sampling Through the Lens of Sequential Decision Making