Graph-based Online Monitoring of Train Driver States via Facial and Skeletal Features

--Driver fatigue poses a significant challenge to railway safety, with traditional systems like the dead-man switch offering limited and basic alertness checks. This study presents an online behavior-based monitoring system utilizing a cus-tomised Directed-Graph Neural Network (DGNN) to classify train driver's states into three categories: alert, not alert, and pathological. T o optimize input representations for the model, an ablation study was performed, comparing three feature configurations: skeletal-only, facial-only, and a combination of both. Experimental results show that combining facial and skeletal features yields the highest accuracy (80.88%) in the three-class model, outperforming models using only facial or skeletal features. Furthermore, this combination achieves over 99% accuracy in the binary alertness classification. Additionally, we introduced a novel dataset that, for the first time, incorporates simulated pathological conditions into train driver monitoring, broadening the scope for assessing risks related to fatigue and health. This work represents a step forward in enhancing railway safety through advanced online monitoring using vision-based technologies. Fatigue is a critical safety concern in railway operations, where long shifts and repetitive activities can significantly impair a driver's alertness [1]. Despite regulations aimed at ensuring adequate rest for train drivers, fatigue-related incidents remain alarmingly common. According to research by the AA Charitable Trust, one in eight drivers admits to falling asleep at the wheel, while nearly two-fifths have felt so tired that they feared they might nod off [2]. The risk is particularly high with modern automated trains operating on night shifts and monotonous routes [3].