Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models