In-Context Learning for Pure Exploration