Post-processing of EEG-based Auditory Attention Decoding Decisions via Hidden Markov Models

--Auditory attention decoding (AAD) algorithms exploit brain signals, such as electroencephalography (EEG), to identify which speaker a listener is focusing on in a multi-speaker environment. While state-of-the-art AAD algorithms can identify the attended speaker on short time windows, their predictions are often too inaccurate for practical use. In this work, we propose augmenting AAD with a hidden Markov model (HMM) that models the temporal structure of attention. More specifically, the HMM relies on the fact that a subject is much less likely to switch attention than to keep attending the same speaker at any moment in time. We show how a HMM can significantly improve existing AAD algorithms in both causal (real-time) and non-causal (offline) settings. We further demonstrate that HMMs outperform existing postprocessing approaches in both accuracy and responsiveness, and explore how various factors such as window length, switching frequency, and AAD accuracy influence overall performance. The proposed method is computationally efficient, intuitive to use and applicable in both real-time and offline settings. Accurately detecting to whom someone wishes to listen is of crucial importance for a wide array of applications. For example, this would allow a hearing aid to determine which speakers should be enhanced or suppressed [1]-[4]. This problem can potentially be solved by decoding the auditory attention from brain signals using electroencephalography (EEG) [5]-[9]. The most common and reliable method to decode attention from the neural response is based on stimulus reconstruction [3], [5]-[7], [10]. This method is based on the observation that the brain tracks attended speech more than unattended speech [11], [12]. The goal is to train a decoder that reconstructs the temporal variations in the attended speech signal (e.g., its amplitude envelope) from the EEG data.