Inductive Bias and Spectral Properties of Single-Head Attention in High Dimensions

Open in new window