An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution

We could use the same approach taken by many research works that generate images and paint the square with a stack of deconvolution (transposed convolution) layers. To test this idea, we created a dataset consisting of randomly placed 9 9 squares on a 64 64 canvas, as shown in Figure 1b. To evaluate how well models generalize, we define two train/test splits: a uniform split, where all possible center locations are randomly divided 80 percent/20 percent into train vs. test sets, and a quadrant split, where the canvas is divided into four quadrants: squares centered in the first three quadrants are put in the train set and squares in the last quadrant in the test set.