Wannous, Hazem
Online hand gesture recognition using Continual Graph Transformers
Slama, Rim, Rabah, Wael, Wannous, Hazem
Online continuous action recognition has emerged as a critical research area due to its practical implications in real-world applications, such as human-computer interaction, healthcare, and robotics. Among various modalities, skeleton-based approaches have gained significant popularity, demonstrating their effectiveness in capturing 3D temporal data while ensuring robustness to environmental variations. However, most existing works focus on segment-based recognition, making them unsuitable for real-time, continuous recognition scenarios. In this paper, we propose a novel online recognition system designed for real-time skeleton sequence streaming. Our approach leverages a hybrid architecture combining Spatial Graph Convolutional Networks (S-GCN) for spatial feature extraction and a Transformer-based Graph Encoder (TGE) for capturing temporal dependencies across frames. Additionally, we introduce a continual learning mechanism to enhance model adaptability to evolving data distributions, ensuring robust recognition in dynamic environments. We evaluate our method on the SHREC'21 benchmark dataset, demonstrating its superior performance in online hand gesture recognition. Our approach not only achieves state-of-the-art accuracy but also significantly reduces false positive rates, making it a compelling solution for real-time applications. The proposed system can be seamlessly integrated into various domains, including human-robot collaboration and assistive technologies, where natural and intuitive interaction is crucial.
Where Is My Mind (looking at)? Predicting Visual Attention from Brain Activity
Delvigne, Victor, Tits, Noé, La Fisca, Luca, Hubens, Nathan, Maiorca, Antoine, Wannous, Hazem, Dutoit, Thierry, Vandeborre, Jean-Philippe
Visual attention estimation is an active field of research at the crossroads of different disciplines: computer vision, artificial intelligence and medicine. One of the most common approaches to estimate a saliency map representing attention is based on the observed images. In this paper, we show that visual attention can be retrieved from EEG acquisition. The results are comparable to traditional predictions from observed images, which is of great interest. For this purpose, a set of signals has been recorded and different models have been developed to study the relationship between visual attention and brain activity. The results are encouraging and comparable with other approaches estimating attention with other modalities. The codes and dataset considered in this paper have been made available at \url{https://figshare.com/s/3e353bd1c621962888ad} to promote research in the field.
Emotion Estimation from EEG -- A Dual Deep Learning Approach Combined with Saliency
Delvigne, Victor, Facchini, Antoine, Wannous, Hazem, Dutoit, Thierry, Ris, Laurence, Vandeborre, Jean-Philippe
Emotion estimation is an active field of research that has an important impact on the interaction between human and computer. Among the different modality to assess emotion, electroencephalogram (EEG) representing the electrical brain activity presented motivating results during the last decade. Emotion estimation from EEG could help in the diagnosis or rehabilitation of certain diseases. In this paper, we propose a dual method considering the physiological knowledge defined by specialists combined with novel deep learning (DL) models initially dedicated to computer vision. The joint learning has been enhanced with model saliency analysis. To present a global approach, the model has been evaluated on four publicly available datasets and achieves similar results to the state-of-theart approaches and outperforming results for two of the proposed datasets with a lower standard deviation that reflects higher stability. For sake of reproducibility, the codes and models proposed in this paper are available at github.com/VDelv/Emotion-EEG.