Pay Less Attention to Function Words for Free Robustness of Vision-Language Models