In particular, the weights are generated based on the covariance matrix Σ which can be decomposed14 into the equation: 15 Σ = γ2UθΛUθT, = γ2 cos θ sin θ sin θcos θ σ21 0 0 σ22 cos θ sin θ sin θcos θ

Apr-25-2026, 06:16:40 GMT–Neural Information Processing Systems 

An image is worth 16x16 words: Transformers for image82 recognition at scale.