Small Singular Values Matter: ARandom Matrix Analysis of Transformer Models
–Neural Information Processing Systems
This work analyzes singular-value spectra of weight matrices in pretrained transformer models to understand how information is stored at both ends of the spectrum.
Neural Information Processing Systems
Jun-22-2026, 23:47:01 GMT