On Erroneous Agreements of CLIP Image Embeddings