When are Lemons Purple? The Concept Association Bias of CLIP