Attention Mechanisms in Deep Learning -- Not So Special Plow

Attention mechanisms are very important in the sense that they are a ubiquitous and are a necessary component of neural machine learning systems. However, for that very same reason, they are not so special. Contrary to what their newfound recognition would suggest, they are not some exotic new device only present in certain "next generation" models. Instead, they are a component of essentially all deep learning models. Giving them an explicit name and identity is certainly useful, but has misrepresented them as more special than they are.