Learning without Forgetting for Vision-Language Models