Structured Unrestricted-Rank Matrices for Parameter Efficient Finetuning

Recent efforts to scale Transformer models have demonstrated rapid progress across a wide range of tasks (Wei at.