LOBG:Less Overfitting for Better Generalization in Vision-Language Model