Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning