Sparsity May Be All You Need: Sparse Random Parameter Adaptation