Basis-Oriented Low-rank Transfer for Few-Shot and Test-Time Adaptation