GIA-MIC: Multimodal Emotion Recognition with Gated Interactive Attention and Modality-Invariant Learning Constraints
He, Jiajun, Mi, Jinyi, Toda, Tomoki
–arXiv.org Artificial Intelligence
Attention-based fusion methods dominate MER research, achieving strong classification performance. However, two key challenges remain: effectively extracting modality-specific features and capturing cross-modal similarities despite distribution differences caused by modality heterogeneity. To address these, we propose a gated interactive attention mechanism to adaptively extract modality-specific features while enhancing emotional information through pairwise interactions. Additionally, we introduce a modality-invariant generator to learn modality-invariant representations and constrain domain shifts by aligning cross-modal similarities. Experiments on IEMO-CAP demonstrate that our method outperforms state-of-the-art MER approaches, achieving W A 80.7% and UA 81.3%.
arXiv.org Artificial Intelligence
Jun-3-2025