Understanding ConvMixer (with a simple PyTorch implementation)

It is still under review and we don't know the real author/authors. Before diving into the architecture of ConvMixer, let us see how the authors got motivated, and how they used the ideas behind existing ideas to make their new model. Convolutional Neural Networks have been dominating the field of computer vision tasks, and now it is the Transformers that are making the buzz. With their very powerful architectural design, transformers have been very successful in the field of NLP, and now they are doing the same thing with vision. The "self-attention" in these vision transformers is quadratic in time O(n²), due to which they work with "patches of images".