Continual Pre-training of MoEs: How robust is your router?