Toward Efficient Inference for Mixture of Experts