Low-Entropy Latent Variables Hurt Out-of-Distribution Performance