Latent-IMH: Efficient Bayesian Inference for Inverse Problems with Approximate Operators