Graph4MM: Weaving Multimodal Learning with Structural Information