Toward Efficient Inference for Mixture of Experts Haiyang Huang

But training is only half the story. MoE inference is important yet challenging as large language models are deployed for production services.