Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models

Open in new window