Goto

Collaborating Authors

 Gao, Xiaorong


Decoding Natural Images from EEG for Object Recognition

arXiv.org Artificial Intelligence

Electroencephalography (EEG) signals, known for the convenient non-invasive acquisition but low signal-to-noise, have recently gained substantial attention due to the potential to decode natural images. This paper presents a self-supervised framework to demonstrate the feasibility of learning image representations from EEG signals, particularly for object recognition. The framework utilizes image and EEG encoders to extract features from paired image stimuli and EEG responses. With the framework, we attain significantly above-chance results on a comprehensive EEG-image dataset, achieving a top-1 accuracy of 15.6% and a top-5 accuracy of 42.8% in challenging 200-way zero-shot tasks. Moreover, we perform extensive experiments to explore the biological plausibility by resolving the temporal, spatial, spectral, and semantic aspects of EEG signals. Besides, we introduce attention modules to capture spatial correlations, providing implicit evidence of the brain activity perceived from EEG data. These findings yield valuable insights for neural decoding and brain-computer interfaces in real-world scenarios. Our daily life relies on accurately and rapidly identifying objects in complex visual environments (DiCarlo & Cox, 2007). Researchers have pursued decoding natural images from brain activity, aiming to deepen our understanding of the brain and create user-friendly brain-computer interfaces (BCIs) (Kamitani & Tong, 2005; Kay et al., 2008; Gao et al., 2021). Functional magnetic resonance imaging (fMRI), which records blood oxygen level-dependent signals, has been a popular choice for categorizing objects observed by humans (Du et al., 2023; Horikawa & Kamitani, 2017; Allen et al., 2022). Despite its high spatial resolution, fMRI typically requires several seconds to provide a stable response to a single stimulus, limiting its real-time applicability in daily interactions (Lin et al., 2022). Similarly, Magnetoencephalogram (MEG), with high time resolution, has been employed for this purpose, but it is hindered by cost and large devices (Cichy et al., 2014; Hebart et al., 2023). Electroencephalogram (EEG) has emerged as a valuable tool for decoding images based on visualevoked brain activities (Spampinato et al., 2017). EEG has high time resolution, low cost, and good portability, but the low signal-to-noise ratio takes problems (Pan et al., 2022; Kobler et al., 2022).


Visual tracking brain computer interface

arXiv.org Artificial Intelligence

Brain-computer interfaces (BCIs) offer a way to interact with computers without relying on physical movements. Non-invasive electroencephalography (EEG)-based visual BCIs, known for efficient speed and calibration ease, face limitations in continuous tasks due to discrete stimulus design and decoding methods. To achieve continuous control, we implemented a novel spatial encoding stimulus paradigm and devised a corresponding projection method to enable continuous modulation of decoded velocity. Subsequently, we conducted experiments involving 17 participants and achieved Fitt's ITR of 0.55 bps for the fixed tracking task and 0.37 bps for the random tracking task. The proposed BCI with a high Fitt's ITR was then integrated into two applications, including painting and gaming. In conclusion, this study proposed a visual BCI-based control method to go beyond discrete commands, allowing natural continuous control based on neural activity.