Vision Language Models Know Law of Conservation without Understanding More-or-Less

Conservation is a critical milestone of cognitive development considered to be supported by both the understanding of quantitative concepts and the reversibility of mental operations. To assess whether this critical component of human intelligence has emerged in Vision Language Models, we have curated the ConserveBench, a battery of 365 cognitive experiments across four dimensions of physical quantities: volume, solid quantity, length, and number. The former two involve only transformational tasks, whereas the latter two involve non-transformational tasks assessing the understanding of quantitative concepts alone. Surprisingly, we find that while Vision Language Models are generally capable of conserving, they tend to fail at non-transformational tasks whose successes are typically considered to be evidence of the ability to conserve. This implies that the law of conservation, at least in concrete domains, may exist without corresponding conceptual understanding of quantity.