Compensation of Fiber Nonlinearities in Digital Coherent Systems Leveraging Long Short-Term Memory Neural Networks

-- We introduce for the first time the utilization of Long short - term memory (LSTM) neural network architectures for the compensation of fiber nonlinearities in digital coherent systems. We conduct numerical simulations considering either C - band or O - band transmission systems for single channel and multi - channel 16 - QAM modulation format with polarization multiplexing . A detailed analysis regarding the effect of the number of hidden units and the length of the word of sym bols that trains the LSTM algorithm and corresponds to the considered channel memory is conducted in order to reveal the limits of LSTM based receiver with respect to performance and complexity. The numerical results show that LSTM Neural Networks can be v ery efficient as post processors of optical receivers which clas sify data that have undergone non - linear impairments in fiber and provide superior performance compared to digital back propagation, especially in the multi - channel transmission scenario. The complexity analysis shows that LSTM becomes more complex as the number of hidden units and the channel memory increase can be less complex than DBP in long distances ( 1000 km). There is a huge effort in fiber - optic communication industry to cope with the exponentially increasing capacity demands coming from next generation mobile networks and high bandwidth internet applica tions [1]. New trends such as internet of things especially in the context of tactile internet increase the requirements for real - time, high bandwidth, high availability connectivity in the access network domain, thus enhancing the capacity needs in metro and long - haul transmission networks. Optical fiber communication community predicted the imminent explosion of capacity needs ten years ago and started working intensively on techniques that can leverage fiber capabilities in this respect.