Triplet Synthesis For Enhancing Composed Image Retrieval via Counterfactual Image Generation

The collection and access large-scale visual data. Construction of the CIR of these triplets is typically costly and traditionally relies on manual model utilizes triplets that consist of a reference image, modification annotation [12,13], which makes it difficult to gather the large-scale text describing desired changes, and a target image that reflects datasets necessary for practical CIR model training. To deal with these changes. For effectively training CIR models, extensive manual this issue, Ventura et al. have proposed an automatic method to annotation to construct high-quality training datasets, which can select image pairs for triplets from captions previously assigned to be time-consuming and labor-intensive, is required. To deal with the large-scale image dataset [14]. However, this automatic triplet this problem, this paper proposes a novel triplet synthesis method by collection method has several critical issues. This method focuses leveraging counterfactual image generation. By controlling visual solely on collecting similar images based on their captions, which feature modifications via counterfactual image generation, our approach may obtain low-quality triplets. That is, the pairs of images for automatically generates diverse training triplets without any triplets differ significantly in aspects not described by the modification manual intervention.