Deep Optimizer States: Towards Scalable Training of Transformer Models Using Interleaved Offloading

Open in new window