The Demon is in Ambiguity: Revisiting Situation Recognition with Single Positive Multi-Label Learning

--Context recognition (SR) is a fundamental task in computer vision that aims to extract structured semantic summaries from images by identifying key events and their associated entities. Specifically, given an input image, the model must first classify the main visual events (verb classification), then identify the participating entities and their semantic roles (semantic role labeling), and finally localize these entities in the image (semantic role localization). Existing methods treat verb classification as a single-label problem, but we show through a comprehensive analysis that this formulation fails to address the inherent ambiguity in visual event recognition, as multiple verb categories may reasonably describe the same image. This paper makes three key contributions: First, we reveal through empirical analysis that verb classification is inherently a multi-label problem due to the ubiquitous semantic overlap between verb categories. Second, given the impracticality of fully annotating large-scale datasets with multiple labels, we propose to reformulate verb classification as a single positive multi-label learning (SPMLL) problem - a novel perspective in SR research. Third, we design a comprehensive multi-label evaluation benchmark for SR that is carefully designed to fairly evaluate model performance in a multi-label setting. T o address the challenges of SPMLL, we futher develop the Graph Enhanced V erb Multilayer Perceptron (GE-V erbMLP), which combines graph neural networks to capture label correlations and adversarial training to optimize decision boundaries. Extensive experiments on real-world datasets show that our approach achieves more than 3% improvement on the more meaningful multi-label A verage Precision (MAP) metric while remaining competitive on traditional top-1 and top-5 accuracy metrics. T o our knowledge, our research is the first work that the formulate, solving, and evaluating of verb classification in the SPMLL fashion, which provides theoretical insights and practical tools for advancing situation recognition research. Modern multimedia applications increasingly demand systems that can understand images at both the object level (recognizing individual entities) and the event level (comprehending interactions and activities). Situation Recognition (SR) has emerged as a crucial task addressing this need by extracting structured semantic representations from images [25], [26].