Streamlining Conformal Information Retrieval via Score Refinement