PARIC: Probabilistic Attention Regularization for Language Guided Image Classification from Pre-trained Vison Language Models