How Language Models Prioritize Contextual Grammatical Cues?