Leveraging Separated World Model for Exploration in Visually Distracted Environments