Capsule Neural Networks – Part 2: What is a Capsule?

In classic CNNs, each neuron in the first layer represents a pixel. Then, it feeds this information forward to next layers. The next convolutional layers group a bunch of neurons together, so that a single neuron there can represent a whole frame (bunch) of neurons. Thus, it can learn to represent a group of pixels that look something like a snout, especially if we have many examples of those in our dataset, and the neural net will learn to increase the weight (importance) of that snout neuron feature when identifying if that image is of a dog. However, this method solely cares about the existence of the object in the picture around a specific location; but it is insensitive to the spatial relations and direction of the object.