Complementarity-driven Representation Learning for Multi-modal Knowledge Graph Completion

Li, Lijian

arXiv.org Artificial Intelligence 

Multi-modal Knowledge Graph Completion (MMKGC) aims to uncover hidden world knowledge in multimodal knowledge graphs by leveraging both multimodal and structural entity information. However, the inherent imbalance in multimodal knowledge graphs, where modality distributions vary across entities, poses challenges in utilizing additional modality data for robust entity representation. Existing MMKGC methods typically rely on attention or gate-based fusion mechanisms but overlook complementarity contained in multi-modal data. In this paper, we propose a novel framework named Mixture of Complementary Modality Experts (MoCME), which consists of a Complementarity-guided Modality Knowledge Fusion (CMKF) module and an Entropy-guided Negative Sampling (EGNS) mechanism. Additionally, we introduce an Entropy-guided Negative Sampling mechanism to dynamically prioritize informative and uncertain negative samples to enhance training effectiveness and model robustness. Extensive experiments on five benchmark datasets demonstrate that our MoCME achieves state-of-the-art performance, surpassing existing approaches. Introduction Knowledge graphs (KGs) [1, 2, 3, 4, 5] model real-world knowledge through structured representations in the form of triples--comprising a head entity, a relation, and a tail entity--which are typically constructed manually based on existing databases. However, the inherent incompleteness of KGs [6, 7], coupled with the high cost of annotating factual triples, has given rise to the task of Knowledge Graph Completion (KGC), which aims to predict and infer missing but plausible triples within an existing knowledge graph. Conventional KGC methods [1, 2, 3, 4] predominantly rely on Knowledge Graph Embedding (KGE) techniques, where entities and relations are embedded into continuous vector spaces to learn structural representations that model the relational patterns of triples and assess their plausibility .