Sync-TVA: A Graph-Attention Framework for Multimodal Emotion Recognition with Cross-Modal Fusion
Deng, Zeyu, Lu, Yanhui, Liao, Jiashu, Wu, Shuang, Wei, Chongfeng
–arXiv.org Artificial Intelligence
--Multimodal emotion recognition (MER) is crucial for enabling emotionally intelligent systems that perceive and respond to human emotions. However, existing methods suffer from limited cross-modal interaction and imbalanced contributions across modalities. T o address these issues, we propose Sync-TV A, an end-to-end graph-attention framework featuring modality-specific dynamic enhancement and structured cross-modal fusion. Our design incorporates a dynamic enhancement module for each modality and constructs heterogeneous cross-modal graphs to model semantic relations across text, audio, and visual features. Experiments on MELD and IEMOCAP demonstrate consistent improvements over state-of-the-art models in both accuracy and weighted F1 score, especially under class-imbalanced conditions. ITH the rapid development of artificial intelligence (AI) and robotics, robots are taking on increasingly prominent roles in various aspects of daily life, and the demand for emotional intelligence has become more critical than ever [1]. Modern computer systems can already capable of exhibiting a degree of empathy and perform sophisticated emotional analysis [2]. For domestic robots, accurately recognising and responding to human emotions is vital for building and sustaining harmonious, long-term relationships--particularly after the initial novelty of interaction fades and the need for continuous re-engagement arises [3]. However, the integration between emotion recognition and facial recognition systems remains insufficient and has yet to result in a unified framework. This is mainly because these two research fields have historically evolved independently, rather than being systematically studied and integrated into a collaborative recognition system [4].
arXiv.org Artificial Intelligence
Jul-30-2025
- Country:
- North America > United States (0.28)
- Genre:
- Research Report > Promising Solution (0.34)
- Industry:
- Energy > Power Industry (0.40)
- Health & Medicine > Therapeutic Area
- Psychiatry/Psychology > Mental Health (0.34)
- Technology:
- Information Technology > Artificial Intelligence
- Robots (1.00)
- Cognitive Science > Emotion (1.00)
- Machine Learning > Neural Networks
- Deep Learning (1.00)
- Information Technology > Artificial Intelligence