Goto

Collaborating Authors

 Deep Learning




Supplementary for: " GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization "

Neural Information Processing Systems

We organize our supplementary document as follows: 1. Results on additional dataset 2. Results for limited data settings on YFCC26k and GWS15k datasets 3. Additional Ablations (a) Gallery Size (b) Queue Length (c) ฯƒฮท for Batch GPS noise (d) ฯƒฮท for Queue GPS noise (e) ฯƒ for Random Fourier Features (f) Number of hierarchies (M) 4. Different selection choices for GPSGallery Construction (a) Evenly Spaced GPSCoordinates (b) Test Set GPSCoordinates 5. Analysis of Runtime and Memory Footprint 6. Motivations for using Pretrained CLIP as Image encoder Backbone 7. Qualitative Demonstration (a) Hierarchical learning in our location encoder L () (b) GeoCLIP with Image Query (c) Distribution of correct predictions of GeoCLIP on different datasets (d) GeoCLIP with Text Query 8. Discussion on Ethical Issues and Possible Mitigation In section 4.1 of the main paper, we demonstrated the performance of our GeoCLIP method on Im2GPS3k [2] and GWS15k [1] datasets and compared them with the state-of-the-art methods. Here, we perform experiments on another dataset YFCC26k [6]. The results are provided in Table 1. This result highlights that GeoCLIP performs well across datasets, being useful across different data distributions. GeoCLIP achieves decent performance across datasets even when the training data is significantly reduced. 2 We show the efficacy of GeoCLIP on limited training samples of Im2GPS3k in section 4.2 of the main paper. Now, we further investigate the performance of GeoCLIP for limited data settings on other datasets (YFCC26k and GWS15k).






Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text Classification

Neural Information Processing Systems

Extreme multi-label text classification (XMC) seeks to find relevant labels from an extreme large label collection for a given text input. Many real-world applications can be formulated as XMC problems, such as recommendation systems, document tagging and semantic search. Recently, transformer based XMC methods, such as XTransformer and LightXML, have shown significant improvement over other XMC methods. Despite leveraging pre-trained transformer models for text representation, the fine-tuning procedure of transformer models on large label space still has lengthy computational time even with powerful GPUs. In this paper, we propose a novel recursive approach, XR-Transformer to accelerate the procedure through recursively fine-tuning transformer models on a series of multi-resolution objectives related to the original XMC objective function. Empirical results show that XRTransformer takes significantly less training time compared to other transformerbased XMC models while yielding better state-of-the-art results. In particular, on the public Amazon-3M dataset with 3 million labels, XR-Transformer is not only 20x faster than X-Transformer but also improves the Precision@1 from 51% to 54%. Our code is publicly available at https://github.com/amzn/pecos.



Object-Centric Slot Diffusion

Neural Information Processing Systems

The recent success of transformer-based image generative models in object-centric learning highlights the importance of powerful image generators for handling complex scenes. However, despite the high expressiveness of diffusion models in image generation, their integration into object-centric learning remains largely unexplored in this domain. In this paper, we explore the feasibility and potential of integrating diffusion models into object-centric learning and investigate the pros and cons of this approach. We introduce Latent Slot Diffusion (LSD), a novel model that serves dual purposes: it is the first object-centric learning model to replace conventional slot decoders with a latent diffusion model conditioned on object slots, and it is also the first unsupervised compositional conditional diffusion model that operates without the need for supervised annotations like text. Through experiments on various object-centric tasks, including the first application of the FFHQ dataset in this field, we demonstrate that LSD significantly outperforms state-of-the-art transformer-based decoders, particularly in more complex scenes, and exhibits superior unsupervised compositional generation quality. In addition, we conduct a preliminary investigation into the integration of pre-trained diffusion models in LSD and demonstrate its effectiveness in real-world image segmentation and generation.