Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models