Label Propagation for Zero-shot Classification with Vision-Language Models