Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth

Open in new window