Perturb and Recover: Fine-tuning for Effective Backdoor Removal from CLIP