Transferable and Principled Efficiency for Open-Vocabulary Segmentation