A Contrastive Learning Approach to Mitigate Bias in Speech Models