Channelformer: Attention based Neural Solution for Wireless Channel Estimation and Effective Online Training

Dianxin Luan, Student Member, IEEE, John Thompson, Fellow, IEEE Institute for Digital Communications, School of Engineering, University of Edinburgh, Edinburgh, EH9 3JL, UK Email address: Dianxin.Luan@ed.ac.uk, john.thompson@ed.ac.uk Abstract In this paper, we propose an encoder-decoder neural architecture (called Channelformer) to achieve improved channel estimation for orthogonal frequency-division multiplexing (OFDM) waveforms in downlink scenarios. The self-attention mechanism is employed to achieve input precoding for the input features before processing them in the decoder. In particular, we implement multi-head attention in the encoder and a residual convolutional neural architecture as the decoder, respectively. We also employ a customized weight-level pruning to slim the trained neural network with a fine-tuning process, which reduces the computational complexity significantly to realize a low complexity and low latency solution. This enables reductions of up to 70% in the parameters, while maintaining an almost identical performance compared with the complete Channelformer. We also propose an effective online training method based on the fifth generation (5G) new radio (NR) configuration for the modern communication systems, which only needs the available information at the receiver for online training. Using industrial standard channel models, the simulations of attention-based solutions show superior estimation performance compared with other candidate neural network methods for channel estimation. For fifth generation (5G) wireless communication systems and beyond, the orthogonal frequency division multiplexing (OFDM) baseband waveform will be retained [1], which requires precise channel state information in order to compensate for the channel distortion and provide robust communication [2]. Conventional channel estimation methods are the least-squares (LS) and minimum mean squared error (MMSE) approaches [3]. However, with the development of modern communication systems, the LS method cannot achieve precise estimation and the implementation of the MMSE method is challenging as the perfect and complete channel statistics cannot be accessed accurately in advance. Moreover, conventional channel estimation solutions [4] [5] also cannot achieve sufficient performance. Meanwhile, artificial intelligence is impacting on the optimization and configuration of 6G [6]. It motivates the researchers in the field of wireless channel estimation to explore neural network solutions for improved performance [7] [8] [9] [10]. Compared with the conventional methods which aim to find the closed-form expression, neural network methods are typical datadriven methods aiming for a satisfactory and local optimum solution.