ELSA -- Enhanced latent spaces for improved collider simulations