SMART: Shot-Aware Multimodal Video Moment Retrieval with Audio-Enhanced MLLM

Abstract--Video Moment Retrieval is a task in video understanding that aims to localize a specific temporal segment in an untrimmed video based on a natural language query. Despite recent progress in moment retrieval from videos using both traditional techniques and Multimodal Large Language Models (MLLM), most existing methods still rely on coarse temporal understanding and a single visual modality, limiting performance on complex videos. T o address this, we introduce Shot-aware Multimodal Audio-enhanced Retrieval of Temporal Segments (SMART), an MLLM-based framework that integrates audio cues and leverages shot-level temporal structure. SMART enriches multimodal representations by combining audio and visual features while applying Shot-aware T oken Compression, which selectively retains high-information tokens within each shot to reduce redundancy and preserve fine-grained temporal details. We also refine prompt design to better utilize audio-visual cues. Evaluations on Charades-ST A and QVHighlights show that SMART achieves significant improvements over state-of-the-art methods, including a 1.61% increase in R1@0.5 and 2.59% gain in R1@0.7 on Charades-ST A. Index T erms--Video Moment Retrieval, T emporal Localization, Audio-Visual Representation Learning, Shot-aware T oken Compression, Shot Boundary Detection, T emporal Reasoning, Multimodal Large Language Models (MLLM), Video Understanding. ITH the rapid growth of video content shared and created on the internet and social media, the ability to efficiently analyze such content has become increasingly important. One key task in this domain is moment retrieval-- the process of identifying the specific temporal segment within a video that best corresponds to a given natural language query.