Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models