Some Musings on Capsule Networks and DLPaper2Code

Don't you look at the CapsNet architecture and wonder... Wouldn't it have been amazing if I had come up with this idea? I mean, it was visible to all of us that pooling seemed just way too convenient amidst everything else about CNNs; just selecting the maximum weight among a specific number of weights and using that in the upcoming layers. Pooling was probably the easiest thing to visualize and understand in the entire architecture, which seemed very crude. But still, only the Godfather of Deep Learning did it again and came up with something brilliant -- adding layers inside existing layers instead of adding more layers i.e nested layers.... giving rise to the Capsule Networks! Improvements in CNNs started in the direction of adding more and more layers, playing with parameters and gradually towards connecting distant layers to each other to make sense out of their outputs once they were concatenated, when it was observed that simply increasing the number of layers also eventually reduces the performance after a certain point.