Reviews: Theoretical Limits of Pipeline Parallel Optimization and Application to Distributed Deep Learning

The reviewers agreed that this paper is a nice contribution to the literature and provides interesting and potentially useful convergence results in the framework of pipeline parallel optimization. The reviewers were impressed by the rebuttal and encourage the authors to incorporate the clarifications therein into the paper.