Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflection