Recent Advances on Machine Learning-aided DSP for Short-reach and Long-haul Optical Communications

This is mostly due to the success of neural networks (NNs) and in particular the technique of deep learning [1]. Deep learning and the accompanying software tools have also found their way into optical communications and are now indispensable tools in the field; ML is now commonly used in all parts of fiber-optical communication networks [2]. ML is already widely used for parameter estimation in optical networks, with the goal of configuring optical network links. Due to their capacity as universal function approximators, ML algorithms and in particular NNs are also often used in the physical layer to replace suboptimal or overly complex digital signal processing (DSP) algorithms in the receiver or transmitter. The use of ML to replace parts of the transmitter or receiver, e.g., as DSP algorithms or to support forward error correction (FEC) decoding, still poses many research challenges, despite the benefits we already see. In particular, standard out-of-the-box ML solutions typically have higher computational complexity than conventional, optimized algorithms. Due to the enormous data rates at which optical communication systems operate, complexity is a major concern. The parallel structure of NNs can lead to straightforward parallelization (as in the ubiquitous graphics processing unit (GPU) implementations), which makes them attractive for implementation in optical transceivers. A future challenge will be the development of ultra-low-complexity hardware platforms with low power dissipation that can be used in highly integrated, high-speed optical transceivers.