Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training

Open in new window