Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples