E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving
Tang, Yihong, Liao, Haicheng, Nie, Tong, He, Junlin, Qu, Ao, Chen, Kehua, Ma, Wei, Li, Zhenning, Sun, Lijun, Xu, Chengzhong
–arXiv.org Artificial Intelligence
End-to-end autonomous driving (AD) systems increasingly adopt vision-language-action (VLA) models, yet they typically ignore the passenger's emotional state, which is central to comfort and AD acceptance. We introduce Open-Domain End-to-End (OD-E2E) autonomous driving, where an autonomous vehicle (AV) must interpret free-form natural-language commands, infer the emotion, and plan a physically feasible trajectory. We propose E3AD, an emotion-aware VLA framework that augments semantic understanding with two cognitively inspired components: a continuous Valence-Arousal-Dominance (VAD) emotion model that captures tone and urgency from language, and a dual-pathway spatial reasoning module that fuses egocentric and allo-centric views for human-like spatial cognition. A consistency-oriented training scheme, combining modality pretraining with preference-based alignment, further enforces coherence between emotional intent and driving actions. Across real-world datasets, E3AD improves visual grounding and waypoint planning and achieves state-of-the-art (SOTA) VAD correlation for emotion estimation. These results show that injecting emotion into VLA-style driving yields more human-aligned grounding, planning, and human-centric feedback.
arXiv.org Artificial Intelligence
Dec-5-2025
- Country:
- Asia
- North America
- Canada > Quebec
- Montreal (0.04)
- United States > Massachusetts (0.04)
- Canada > Quebec
- Oceania > Australia
- Genre:
- Research Report > New Finding (0.66)
- Industry:
- Automobiles & Trucks (1.00)
- Information Technology > Robotics & Automation (0.92)
- Transportation > Ground
- Road (1.00)
- Technology:
- Information Technology > Artificial Intelligence
- Cognitive Science (1.00)
- Natural Language (1.00)
- Representation & Reasoning (1.00)
- Robots > Autonomous Vehicles (1.00)
- Vision (1.00)
- Information Technology > Artificial Intelligence