Prompt-tuning latent diffusion models for inverse problems