Calibrated Decision-Making through LLM-Assisted Retrieval