ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models