Identity Inference from CLIP Models using Only Textual Data