Supplementary Material

We provide details omitted in the main paper. Another way to understand the coarse-to-fine framework we proposed is from the optimization perspective. The relaxation may lead to sub-optimal solutions. Here we provide the summary of the IDOL algorithm. Second, we refine the coarse indexes with cycle-consistency as shown in algorithm 2. The refinement is decomposed into several steps, gradually discovering the next intermediate domain in sequence.