Infusion: Preventing Customized Text-to-Image Diffusion from Overfitting