SCRAPS: Speech Contrastive Representations of Acoustic and Phonetic Spaces