Algebraic Invariants of Lightning Self-Attention

Alexandr, Yulia, Duan, Hao, Montúfar, Guido

arXiv.org Machine Learning 

We study the polynomial coefficients of lightning self-attention as coordinates of an algebraic variety. We identify linear and nonlinear families of algebraic invariants, including Chow-type, low-rank, Veronese-type, and Sylvester resultant-based constraints.

Duplicate Docs Excel Report

Title
None found

Similar Docs  Excel Report  more

TitleSimilaritySource
None found