Parameter-Efficient Fine-Tuning without Introducing New Latency