Explore to Generalize in Zero-Shot RL