What is Attention?

Attention is becoming increasingly popular in machine learning, but what makes it such an attractive concept? What is the relationship between attention as applied in artificial neural networks, and its biological counterpart? What are the components that one would expect to form an attention-based system in machine learning? In this tutorial, you will discover an overview of attention and its application in machine learning. Photo by Rod Long, some rights reserved.