PARC: A Quantitative Framework Uncovering the Symmetries within Vision Language Models