Image deduplication using OpenAI's CLIP and Community Detection

A short guide on how to use image embeddings from OpenAI's CLIP and clustering techniques in order to group near-duplicate images together. CLIP is trained by trying to align image text embedding pairs, or "learning visual representations from natural language supervision". You can use it's text or image embeddings to accomplish a lot of different tasks, such as zero-shot image classification! It's embeddings are pretty powerful. For this task, we're going to use the AirBnB Duplicate Image Dataset, available on Kaggle.