Flow Matching for Posterior Inference with Simulator Feedback