ToViLaG: Your Visual-Language Generative Model is Also An Evildoer