Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis