Reviews: GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism

While the proposed pipelining framework is simple, the reviewers unanimously assert that from an empirical point of view it is an important advance, especially in language modeling tasks.