Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

Open in new window