How do CNNs Deal with Position Differences?

An engineer who's learning about using convolutional neural networks for image classification just asked me an interesting question; how does a model know how to recognize objects in different positions in an image? Since this actually requires quite a lot of explanation, I decided to write up my notes here in case they help some other people too. Here's two example images showing the problem that my friend was referring to: If you're trying to recognize all images with the sun shape in them, how do you make sure that the model works even if the sun can be at any position in the image? It's an interesting problem because there are really three stages of enlightenment in how you perceive it: My friend is at the third stage of enlightenment, but is smart enough to realize that there are few accessible explanations of why CNNs cope so well. I don't claim to have any novel insights myself, but over the last few years of working with image models I have picked up some ideas from experience, and heard folklore passed down through the academic family tree, so I want to share what I know.