Speaker Retrieval in the Wild: Challenges, Effectiveness and Robustness