Overwriting Pretrained Bias with Finetuning Data