We present a simple, efficient "direct learning" approach to train Volterra series-based digital pre-distortion filters using neural networks. We show its superior performance over conventional training methods using a 64-QAM 64-GBaud simulated transmitter with varying transmitter nonlinearity and noisy conditions.

For the physical layer, deep learning has shown a great potential either to optimize individual digital signal processing (DSP) blocks or, more interestingly, to optimize the whole DSP blocks as a cascade of neural networks (NNs), referred to as an autoencoder. To mention examples of the former approach, NN-based digital pre-distortion has been proposed to pre-compensate the nonlinear impairments of optical coherent transmitters [4,5], outperforming other pre-distortion techniques such as Volterra-series based solutions. At the receiver side, different NN-based equalizations have been proposed to mitigate optical fiber and transceiver nonlinearities [6-8]. Another example is the use of deep neural networks for digital back-propagation with optimized computational complexity [9]. In the end-to-end (E2E) learning approach, the transmitter, channel, and receiver of a communication system are implemented as an autoencoder, jointly trained to best match the outputs to the inputs and thus, to better communicate [10]. In optical fiber communications, the E2E learning has been applied for both intensity modulation with direct detection (IM/DD) systems [11] and coherent systems [12-15]. For the latter, E2E learning is more involved due to the complex interplay of nonlinearity and chromatic dispersion in optical fibers. For coded-modulation systems, most of E2E learning studies focused so far on geometric constellation shaping, e.g.

In this paper, we implement an optical fiber communication system as an end-to-end deep neural network, including the complete chain of transmitter, channel model, and receiver. This approach enables the optimization of the transceiver in a single end-to-end process. We illustrate the benefits of this method by applying it to intensity modulation/direct detection (IM/DD) systems and show that we can achieve bit error rates below the 6.7\% hard-decision forward error correction (HD-FEC) threshold. We model all componentry of the transmitter and receiver, as well as the fiber channel, and apply deep learning to find transmitter and receiver configurations minimizing the symbol error rate. We propose and verify in simulations a training method that yields robust and flexible transceivers that allow---without reconfiguration---reliable transmission over a large range of link dispersions. The results from end-to-end deep learning are successfully verified for the first time in an experiment. In particular, we achieve information rates of 42\,Gb/s below the HD-FEC threshold at distances beyond 40\,km. We find that our results outperform conventional IM/DD solutions based on 2 and 4 level pulse amplitude modulation (PAM2/PAM4) with feedforward equalization (FFE) at the receiver. Our study is the first step towards end-to-end deep learning-based optimization of optical fiber communication systems.