Sample-Specific Debiasing for Better Image-Text Models