I want to calculate certain pixel property that is based on it's neighbourhood • /r/MachineLearning

This is basically the same setting as semantic segmentation so any architecture that solves this problem can be adapted for your application. You can have a look at papers like U-Net, or Fully Convolutional Neural Networks, whose implementations are available for Caffe (I would also be surprised if there weren't re-implementations in other frameworks like TensorFlow or Theano). On a side note, I am personally using the very naive approach that you are describing (extracting patches around each pixel) and I have found that the performance loss is not that critical for my application. It might still be a good idea for you to try it anyway before switching to more "exotic" architectures.