Geometrically Enriched Latent Spaces