In this paper, we investigate the use of the learned digital back-propagation (LDBP) for equalizing dual-polarization fiber-optic transmission in dispersion-managed (DM) links. LDBP is a deep neural network that optimizes the parameters of DBP using the stochastic gradient descent. We evaluate DBP and LDBP in a simulated WDM dual-polarization fiber transmission system operating at the bitrate of 256 Gbit/s per channel, with a dispersion map designed for a 2016 km link with 15% residual dispersion. Our results show that in single-channel transmission, LDBP achieves an effective signal-to-noise ratio improvement of 6.3 dB and 2.5 dB, respectively, over linear equalization and DBP. In WDM transmission, the corresponding $Q$-factor gains are 1.1 dB and 0.4 dB, respectively. Additionally, we conduct a complexity analysis, which reveals that a frequency-domain implementation of LDBP and DBP is more favorable in terms of complexity than the time-domain implementation. These findings demonstrate the effectiveness of LDBP in mitigating the nonlinear effects in DM fiber-optic transmission systems.

Mitigation of these distortions is possible using, e.g., digital back-propagation (DBP) [1]. DBP, however, requires knowledge of fiber parameters and topology, and can be computationally expensive in part due to potentially large number of spatial segments. Artificial neural networks (ANNs) offer an alternative approach that might be less complex [2]. Learned digital backpropagation (LDBP) is proposed in [3], in which the model is based on the split-step Fourier method (SSFM), and optimized using the standard learning algorithms for ANNs. In this paper, we consider a convolutional neural network (CNN) equalizer. We note that the SSFM coefficients are repeated in each span. Thus, instead of joint training of all neural network layers, we train a few unique layers that are shared in network depth. With this parameter sharing method, we substantially reduce the number of model trainable parameters.

An important problem in fiber-optic communications is to invert the nonlinear Schr\"odinger equation in real time to reverse the deterministic effects of the channel. Interestingly, the popular split-step Fourier method (SSFM) leads to a computation graph that is reminiscent of a deep neural network. This observation allows one to leverage tools from machine learning to reduce complexity. In particular, the main disadvantage of the SSFM is that its complexity using M steps is at least M times larger than a linear equalizer. This is because the linear SSFM operator is a dense matrix. In previous work, truncation methods such as frequency sampling, wavelets, or least-squares have been used to obtain "cheaper" operators that can be implemented using filters. However, a large number of filter taps are typically required to limit truncation errors. For example, Ip and Kahn showed that for a 10 Gbaud signal and 2000 km optical link, a truncated SSFM with 25 steps would require 70-tap filters in each step and 100 times more operations than linear equalization. We find that, by jointly optimizing all filters with deep learning, the complexity can be reduced significantly for similar accuracy. Using optimized 5-tap and 3-tap filters in an alternating fashion, one requires only around 2-6 times the complexity of linear equalization, depending on the implementation.

A neural-network-based approach is presented to efficiently implement digital backpropagation (DBP). For a 32x100 km fiber-optic link, the resulting "learned" DBP significantly reduces the complexity compared to conventional DBP implementations.