A guide to convolution arithmetic for deep learning

Although CNNs have been used as early as the nineties to solve character recognition tasks (Le Cunet al., 1997), their current widespread application is due to much more recent work, when a deep CNN was used to beat state-of-the-art in the ImageNet image classification challenge (Krizhevsky et al., 2012). Convolutional neural networks therefore constitute a very useful tool for machine learning practitioners. However, learning to use CNNs for the first time is generally an intimidating experience. A convolutional layer's output shape is affected by the shape of its input as well as the choice of kernel shape, zero padding and strides, and the relationship between these properties is not trivial to infer. This contrasts with fully-connected layers, whose output size is independent of the input size. Additionally, CNNs also usually feature apool-ing stage, adding yet another level of complexity with respect to fully-connected networks. Finally, so-called transposed convolutional layers (also known as fractionally strided convolutional layers) have been employed in more and more work as of late (Zeileret al., 2011; Zeiler and Fergus, 2014; Longet al., 2015; Rad-ford et al., 2015; Visinet al., 2015; Imet al., 2016), and their relationship with convolutional layers has been explained with various degrees of clarity. This guide's objective is twofold: 1. Explain the relationship between convolutional layers and transposed con-volutional layers.