What Happens During Finetuning of Vision Transformers: An Invariance Based Investigation