r/MachineLearning - [D] Activation Function as Attention

Focusing on the center would be something like a gaussian activation function (which i didn't see before but I think is used in some articles). But I'm not sure how they can be seen as attention. Attention is commonly seen as focusing on one part of the input among several (for example one image region, or one word) and is usually implemented by giving a score to each part, then softmax this score, and average all inputs with those scores, giving you a weighted representation of your inputs. That doesn't have much to do with the activation, does it?