Demystifying "Matrix Capsules with EM Routing." Part 1: Overview

Recently, Geoffrey Hinton, one of the fathers of deep learning, made waves in the machine learning community by publishing a revolutionary computer vision architecture: capsule networks. Hinton has been pushing for using capsule networks since 2012, after he first revolutionized the use of Convolutional Neural Networks (CNNs) for image detection, but only now has he made them feasible. The initial successful approach, published two weeks ago, is titled "Dynamic Routing Between Capsules." Dynamic routing -- which we'll be exploring in depth throughout this post -- allows networks to more intuitively understand part-whole relationships. In the three days following the release of this paper, another paper on dynamic routing in capsule networks was submitted for review to ICLR 2018. This paper, titled "Matrix Capsules for EM Routing," is widely speculated to have been authored by Hinton, and discusses a revolutionary new method for dynamic routing -- even compared to his first paper.