When and Where do Data Poisons Attack Textual Inversion?