Continual Pre-training of MoEs: How robust is your router?

Open in new window