Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling

However, traditional neural network blocks aim to learn the feature representations in a local sense. For example, both convolutional and recurrent operations process a local neighborhood (several nearest neighboring neurons) in either space or time.