Focal Attention for Long-Range Interactions in Vision Transformers Jianwei Y ang

Recently, Vision Transformer and its variants have shown great promise on various computer vision tasks. The ability of capturing local and global visual dependencies through self-attention is the key to its success.