Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation