Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers