Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure