Attention in Computer Vision, Part 3: GE

In this article, gather-excite will be discussed, an attention mechanism that aggregates information from large receptive fields and redistributes it to local features for expressing long-range spatial interactions. You can find the GitHub repository for this article here. Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks opens by noting that, in theory, the receptive field of convolutional neural networks is sufficiently extensive to cover the totality of input images. However, the effective receptive field in practice is much smaller and not global. The effective receptive field of a 5-layered convolutional neural network with 3 X 3 kernels, for example, encompasses the centre of the theoretical receptive field, but the edges and corners are not included.