Multimodal Large Language Models (MLLMs) demonstrate remarkable fluency in understanding visual scenes, yet they exhibit a critical lack in a fundamental cognitive skill: object counting. This blind spot severely limits their reliability in real-world applications. To date, this capability has been largely unevaluated in complex scenarios, as existing benchmarks either feature sparse object densities or are confined to specific visual domains, failing to test models under realistic conditions. Addressing this gap, we introduce CountQA, a challenging new benchmark designed to probe this deficiency. Comprising over 1,500 question-answer pairs, CountQA features real-world images with high object density, clutter, and occlusion. We investigate this weakness by evaluating 15 prominent MLLMs on the CountQA benchmark and reveal that the top-performing model achieves a mere 42.9% accuracy, with performance declining as object counts rise. By providing a dedicated benchmark to diagnose and rectify this core weakness, CountQA paves the way for a new generation of MLLMs that are not only descriptively fluent but also numerically grounded and spatially aware. We will open-source the dataset and code upon paper acceptance to foster further research.

This article is part of our November 2018 series about the state of retail. Artificial intelligence is reinventing the retail industry as we know it. From personalized customer experiences across digital touch points to improved product management, this powerful area of computing is helping retailers up their relevance, efficiency, and, ultimately, their bottom line. "When we look across the retail industry, there is, surprisingly, quite a broad use of AI already," said Vish Ganapathy, managing director and global retail technology lead at Accenture. "A lot of that has got to do with the fact that more and more technology vendors are injecting AI into everything that they do within their applications. For retail, there are very specific areas where AI can really make a big difference."