Safe Vision-Language Models via Unsafe Weights Manipulation