Deep Neural Network Capsules

A recent paper on Capsules has many important insights for revolutionizing learning in Deep Neural Networks. Here is a picture of CapsNet, the neural network architecture using Capsules. The interesting dynamic routing occurs between PrimaryCaps and DigitCaps. Dynamic routing is implemented with two main transformation as reported in these equations (2 in paper). U are the outputs of Capsules in the layer below, and S are outputs from Capsules on layer above.