Investigating the Invertibility of Multimodal Latent Spaces: Limitations of Optimization-Based Methods