Dual Decomposition of Weights and Singular Value Low Rank Adaptation