Implicit Inversion turns CLIP into a Decoder